[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[EP-tech] Re: Antwort: indexer stalls



Il 03/02/2016 11:13, martin.braendle at id.uzh.ch ha scritto:
>
> Hi Yuri,
>
> - EPrints 3.1.2 or 3.1.12?
>

EPrints 3.1.2.1 (Chocolate-coated Coffee Bean)
>
> - I don't think the large row counts of the two tables are a problem - 
> such counts are quite common.
> - Our experience is that the indexer rather is slowed down by too many 
> entries in the event table, especially if there are some which are 
> ancient (several weeks or months old). See Manage records > Tasks .
>
I don't have such "Manage records > Tasks" (events has been introduced 
in 3.2?), but i think I can just empty the index_queue. The problem is 
that high load it produces.

> Removing old tasks will help a lot.
>

Why the number of entries in the event table should slow the indexer? 
:-/ it processes one by one.

>
> Best regards,
>
> Martin
>
> --
> Dr. Martin Br?ndle
> Zentrale Informatik
> Universit?t Z?rich
> Stampfenbachstr. 73
> CH-8006 Z?rich
>
> mail: martin.braendle at id.uzh.ch
> phone: +41 44 63 56705
> fax: +41 44 63 54505
> http://www.zi.uzh.ch
>
> Inactive hide details for Yuri ---03/02/2016 09:58:47---Hi all! I've a 
> big problem with the eprints indexer (Eprint 3.1.2). IYuri 
> ---03/02/2016 09:58:47---Hi all!   I've a big problem with the eprints 
> indexer (Eprint 3.1.2). I was
>
> Von: Yuri <yurj at alfa.it>
> An: "EPrints.org Technical List" <eprints-tech at ecs.soton.ac.uk>
> Datum: 03/02/2016 09:58
> Betreff: [EP-tech]  indexer stalls
> Gesendet von: eprints-tech-bounces at ecs.soton.ac.uk
>
> ------------------------------------------------------------------------
>
>
>
> Hi all!
>
>  I've a big problem with the eprints indexer (Eprint 3.1.2). I was
> looking for a high load problem in our server and discovered that, even
> with no http requests or other process running, the load was above 1.
>
> The indexer was also at Ss state, so sleeping and log (with an increased
> log level) showed it was not doing any operation in the past days. Then
> I shut down the indexer and the high load disappeared.
>
> I think the problem could be my eprint__rindex which has 33 millions
> rows and eprint__index has 3 million rows. So I think what happens is
> that mysql issue some kernel calls, because I can see that, when the log
> stops to be updated with new operations), the wa (i/o waiting) is above
> 50% for a lot of time.
>
> Then the indexer stop doing indexing and load is high (above 2 or 3)
> with an high wa (I/O).
>
> Now I've stopped the indexer and running it as the webserver user
> (www-data) with --once and --notdeamon and stopping it when stalling.
> Then I run it again and so on (the index_queue has still 4000 rows to
> process).
>
> Any idea on how to solve this problem?
>
> Question: which part of Eprints now is responsible to add an entry in
> the index_queue table?
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: http://www.eprints.org/tech.php/
> *** EPrints community wiki: http://wiki.eprints.org/
> *** EPrints developers Forum: http://forum.eprints.org/
>
>
>
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: http://www.eprints.org/tech.php/
> *** EPrints community wiki: http://wiki.eprints.org/
> *** EPrints developers Forum: http://forum.eprints.org/