[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[EP-tech] Re: load throttling strategy

I confess I just throttled the server: in httpd.conf add

StartServers    2
MinSpareServers 1
MaxSpareServers 1
MaxClients    128

(I put mine right next to where I defined the user & group for the server)

On 15/12/14 15:35, Ian Stuart wrote:
> I bet he's suffering the problem I had: Google loads a 'view' page,
> which has hit it's "out of date" time so rebuilds.
> In the mean time, another httpd catches the next google page load, which
> has also hit its "out of date" time, so spawns ANOTHER page rebuild.
> It very quickly ramps up, and WILL swamp the server... and WILL crash
> the server.
> On 15/12/14 15:21, sf2 wrote:
>> Which issue are you trying to solve by introducing such a delay?
>> Seb
>> On 15.12.2014 15:17, Gooding R.W. wrote:
>>> Hi,
>>> does anyone have an easy way to throttle eprints website crawlers ?
>>> I?m wondering is something as a short delay for each ePrint accessed -
>>> the delay being proportional to the current load ?  That?s perhaps a
>>> little blunt but might it be very easy to implement ?


Ian Stuart.
Developer: ORI, RJ-Broker, and OpenDepot.org
Bibliographics and Multimedia Service Delivery team,
The University of Edinburgh.


This email was sent via the University of Edinburgh.

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.