[Abcde-users] running multiple abcde/cdparanoia in parallel - performance issues

[Charles Steinkuehler]

On 5/31/2015 4:00 PM, Arthur Lutz wrote:
> On 27/05/2015 13:29, Johannes Gernemann wrote:
> 
> Am thinking that if there is no benefit from running them in parallel I
> might as well find a way of getting the to run sequentially. I want to
> have a web frontend that provides information on progress and am looking
> at http://blog.miguelgrinberg.com/post/using-celery-with-flask and
> thinking of running the jobs through celery and viewing the progress
> with flask (am more at ease with python than other programming languages).
> 
>> So how long does it actually take you to rip and encode (what format are
>> you encoding?) let's say 4 CD's in parallel?
> 
> Am encoding to FLAC (which shouldn't be too CPU intensive right?).
> 
> Right now, am having trouble getting through parallel RIPs, they seem to
> always fail. At some point I remember most drives popping out
> successfully after a couple of hours.

I have a few suggestions:

* Put each of your CD drives on it's own IDE bus.  Given the way IDE
works, and the craziness I've seen ripping various CDs, you're asking
for trouble if you share IDE connectors.  Maybe your system is
different, but when I rip some CDs, I get lots of IDE resets and
general wonky behavior from the CD drives quite often.  You want to
isolate that so it doesn't affect other drives.

* If you can't put all your devices on dedicated IDE buses, at least
make sure the HDD(s) do not share an IDE cable with any CD drives
actively ripping.

* Rip to an uncompressed format (like WAV), then queue a separate
low-priority process to compress to your desired format(s).  Any sort
of compression takes a lot of CPU cycles, and I suspect that's what is
really slowing down your rips.

* You might want to run some baseline tests with cdparanoia.  Test
ripping multiple CDs simultaneously outside of abcde to see what your
basic hardware is capable of.  If you can't match this performance in
abcde, you likely need to either change your ripping options, or
perhaps tweak the abcde code.  If you still get lousy performance, you
either need better hardware or a better low-level audio ripping utility.

-- 
Charles Steinkuehler
charles at steinkuehler.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL: <http://lists.einval.com/pipermail/abcde-users/attachments/20150531/b8d26c75/attachment.sig>