[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[EP-tech] indexer stalls



Hi all!

  I've a big problem with the eprints indexer (Eprint 3.1.2). I was 
looking for a high load problem in our server and discovered that, even 
with no http requests or other process running, the load was above 1.

The indexer was also at Ss state, so sleeping and log (with an increased 
log level) showed it was not doing any operation in the past days. Then 
I shut down the indexer and the high load disappeared.

I think the problem could be my eprint__rindex which has 33 millions 
rows and eprint__index has 3 million rows. So I think what happens is 
that mysql issue some kernel calls, because I can see that, when the log 
stops to be updated with new operations), the wa (i/o waiting) is above 
50% for a lot of time.

Then the indexer stop doing indexing and load is high (above 2 or 3) 
with an high wa (I/O).

Now I've stopped the indexer and running it as the webserver user 
(www-data) with --once and --notdeamon and stopping it when stalling. 
Then I run it again and so on (the index_queue has still 4000 rows to 
process).

Any idea on how to solve this problem?

Question: which part of Eprints now is responsible to add an entry in 
the index_queue table?