[Expo-tech] speeding up import about 7x (single user)

Philip Sargent (Gmail) philip.sargent at gmail.com
Wed Apr 15 12:27:58 BST 2020


Aha.
For single user testing with sqlite, simply specify
        'NAME' : ':memory:',     # Or path to database file if using sqlite3.
in localsettings.py

It goes like lightning: 6 minutes instead of about 40 minutes.
So that speeds up testing import parsers really quite usefully.

Of course the memory is cleared afterwards.

Which is odd as the default sqlite cache size is 2MB, which is 20% of a full database.
So doing it entirely using :memory: shouldn't be that fast.

https://www.sqlite.org/inmemorydb.html

Hmm. Maybe this could be used as a way of doing bulk import into staging memory sqlite and then doing a SQL update into the real MySQL database. Hmm.




More information about the Expo-tech mailing list