Mdthbs

I have a process that returns an unpredictable number of data points, and I'd like to run it repeatedly until I have a certain number of points.

My actual code is too complicated to use an illustration, so I wrote this toy example. fakeData[] will return 1-21 data points, and I want to run it until I have at least 100. But this code doesn't work because you can't take the so-far Length[] of a list that you're still building.

fakeData[n_] := RandomReal[1, 1 + RandomInteger[n]];

big = Reap[

   While[Length[big] < 100, (* this doesn't work*)

    Sow[fakeData[20]]]][[2, 1]]

I could just allocate 'big' as a Table with length 100 and copy each new small list into it, but then I'd have to discard some perfectly good data points I laboriously calculated, which is distasteful. Is there a better way?

How about:

SeedRandom[1]

Reap[NestWhile[Join[#, Sow@fakeData[20]] &, {}, LessThan[100]@*Length]][[2, 1]]

{{0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649}, {0.337261,
0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979,
0.964289}, {0.741987, 0.819242}, {0.539713}, {0.012502, 0.439595, 0.169709,
0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254,
0.85584}, {0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108,
0.203219, 0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994,
0.439824, 0.245655, 0.562932}, {0.370393, 0.934574, 0.550753, 0.136193,
0.390665, 0.941924, 0.743334, 0.296465}, {0.114065, 0.612737, 0.596194,
0.32461, 0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871,
0.188398, 0.129602}, {0.265823, 0.750065, 0.757875, 0.679856, 0.0740267,
0.691003, 0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246,
0.578542, 0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029,
0.122614}, {0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315,
0.346533, 0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529,
0.208461, 0.315747, 0.367579, 0.521331, 0.36944, 0.566759}}

Another similar possibility:

SeedRandom[1]

Reap[NestWhile[Length @ Sow @ fakeData[20] &, 0, LessThan[100] @* Plus, All]][[2, 1]]

same answer

The straight forward solution is to simply count the number of points you have sown, i.e.:

big = Module[

    {count = 0},

    Reap[

        While[ count < 100, count += Length@Sow[fakeData[20]] ]

    ][[2,1]]

]

Here's a method that just uses Bag since I think effectively that's what Reap and Sow are using. It's probably a bit slower than adding the lists directly and flattening after, but it's conceptually how you were thinking about the original problem:

bag = Internal`Bag[];

SeedRandom[1]

While[Internal`BagLength[bag] < 100,

  Internal`StuffBag[bag, #] & /@ fakeData[20]

  ];

Internal`BagPart[bag, All]



{0.00683794, 0.0936818, 0.474619, 0.310422, 0.153631, 0.31649, 0.337261, 

0.470877, 0.32728, 0.124887, 0.113682, 0.988692, 0.970078, 0.908979, 

0.964289, 0.741987, 0.819242, 0.539713, 0.012502, 0.439595, 0.169709, 

0.771071, 0.998221, 0.179295, 0.901812, 0.661701, 0.162254, 0.85584, 

0.00132041, 0.784942, 0.693806, 0.687592, 0.525913, 0.842108, 0.203219, 

0.495244, 0.909835, 0.464522, 0.115059, 0.443676, 0.712994, 0.439824, 

0.245655, 0.562932, 0.370393, 0.934574, 0.550753, 0.136193, 0.390665, 

0.941924, 0.743334, 0.296465, 0.114065, 0.612737, 0.596194, 0.32461, 

0.713441, 0.225573, 0.387218, 0.55637, 0.336226, 0.90315, 0.333871, 0.188398, 

0.129602, 0.265823, 0.750065, 0.757875, 0.679856, 0.0740267, 0.691003, 

0.571181, 0.921954, 0.559011, 0.341209, 0.757399, 0.856246, 0.578542, 

0.866321, 0.641392, 0.474307, 0.197374, 0.172371, 0.448029, 0.122614, 

0.146429, 0.0648023, 0.514557, 0.320289, 0.510485, 0.00828315, 0.346533, 

0.0588742, 0.436849, 0.305532, 0.767718, 0.254158, 0.345529, 0.208461, 

0.315747, 0.367579, 0.521331, 0.36944, 0.566759}