[Expo-tech] Memory

[Wookey]

On 2019-07-11 04:29 +0100, Wookey wrote:
> 787MB (648MB drv res) 65.1%
> 
> I've left it running, and maybe it'll finish?

Nope, still ran out of memory eventually at the same 'Loading Pos...' spot.

I could try asking for another GB which would no doubt make it work,
and that might be OK for the duration of expo, but I know the memory
is somewhat constrained so if we could make our script a less
profligate with the RAM that would be better, and it looks like there
is plenty of room for that.

At the loading Pos spot it's finished the survex parsing, but has used
nearly all the memory in the process (largely by reading everything in
twice) then runs survex to process the full dataset into a pos file
(which of course involves reading it all in a 3rd time). Just doing
that survex process at the start instead (order doesn't matter here)
might be sufficient to make this work for now.

Looks like cavern uses 53MB to process 1623-and-1626.svx (which is a
lot sleeker than 500MB), but enough to push this over the 'no memory
left' threshold. 

I've install acct (BSD process accounting) so we can see how much time
and memory things use after the fact.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.wookware.org/pipermail/expo-tech/attachments/20190711/f436a287/attachment.sig>