EPrints Technical Mailing List Archive

Message: #06504

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] IRStats2: process_stats job with InnoDB

I think so - at least it did for us.

Using the transactions halved the time it took our stats to run.

 

From memory, I *think* that I could have been more brutal with the transactions. I think there's a loop outside the begin/commit additions I made.

If the begin/commit were added outside this loop instead, there may be even bigger gains - but I was happy with halving the runtime!

 

Cheers,

John

 

From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of George Macgregor
Sent: 16 May 2017 12:55
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] IRStats2: process_stats job with InnoDB

 

Thanks as always, John!

 

So, making the updates to Handler.pm and Sets.pm should oil the wheels? I’m desperate to kill this current job!

 

Cheers


George

 

From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of John Salter
Sent: 16 May 2017 12:22
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] IRStats2: process_stats job with InnoDB

 

Hi George,

See: https://github.com/eprints/irstats2/pull/84

 

NB There are two forks of IRStats2:

-          https://github.com/eprints/irstats2/

-          https://github.com/eprintsug/irstats2/

I'm not sure which is the most up-to-date. Can anyone comment on this?

Should we standardise on one of them?

 

Cheers,

John

 

From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of George Macgregor
Sent: 16 May 2017 12:10
To: eprints-tech@ecs.soton.ac.uk
Subject: [EP-tech] IRStats2: process_stats job with InnoDB

 

Hello colleagues!

 

I am looking for some IRStats2 insights. We have recently shifted our MySQL DB to InnoDB from MyISAM. I am in the process of running the process_stats job for IRStats2 in order to reflect local changes but the job is taking an aeon – and significantly longer than it ever did under MyISAM.

 

I was informed that it would run more slowing with InnoDB but I had no idea it would be quite so bad. So far it has been running for the best part of a week, which has been impacting on end users (e.g. sporadic unavailability of EPrints, slow responses, etc.). It is so bad that I am almost tempted to kill the job, even though it has been running for the best part of a week.

 

Has a similar issue affected others and, if so, do you have any tips for speeding up the processing? Or do I simply need to take the hit?!

 

Cheers


George

--

George Macgregor

Information Services Directorate | University of Strathclyde

Curran Building | 101 St James Road

Glasgow G4 ONS

Tel: 0141 548 3496

Email: george.macgregor@strath.ac.uk

Web: https://purl.org/g3om4c

Twitter: @g3om4c

iD iconorcid.org/0000-0002-8482-3973
--

Ensure that your research outputs are eligible for submission in the next REF.  Authors' accepted manuscripts of journal articles and conference proceedings accepted after 01 April 2016 must be deposited in PURE as soon as possible after acceptance for publication – email openaccess@strath.ac.uk for further information or visit http://www.strath.ac.uk/openaccess.

 

twitter logo

--

The University of Strathclyde is a charitable body, registered in Scotland, with registration number SC015263