[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[EP-tech] New system, crashing periodically b/c MySQL Locking



Hi Fran,
A couple of things to look at:
In ~/perl_lib/EPrints/Database/mysql.pm, do you have a line that references 'optimizer_search_depth'?
If not, this might be useful: https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feprints%2Feprints%2Fissues%2F120&data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cf0d6d6cec0d2498f630908d692911ac8%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&sdata=IFVJuzMFWLnSX%2B8h12S1JbrFiBm56nBfCMZ8y%2Fij0aw%3D&reserved=0


If you already have that, what about the storage engines the tables are using?
MyISAM uses table-level locking, but InnoDB use row-level locking.
I did some investigations a while ago, and found that some queries that looked like they shouldn't lock a table were doing so for some reason (I cannot recall the exact details - but I don't think it's an 'eprints' thing - I could recreate the scenario in the DB directly).

Any of that help?
Cheers,
John


From: eprints-tech-bounces at ecs.soton.ac.uk [mailto:eprints-tech-bounces at ecs.soton.ac.uk] On Behalf Of Fran Callaghan via Eprints-tech
Sent: 14 February 2019 14:50
To: eprints-tech at ecs.soton.ac.uk
Subject: [EP-tech] New system, crashing periodically b/c MySQL Locking

Hi Everybody,

We have a (reasonably) new install that has been grinding to a halt every week or two. When I look at 'top' on the command line (Red Hat) mysqld is hogging 100% CPU. When I use mysqladmin processlist I can see a number of sessions queuing behind this...

| 104558 | doras | localhost | doras | Query   | 1587 | statistics                   | INSERT INTO `cache17312`(`pos`, `eprintid`) SELECT @i:=@i+1, `eprintid` FROM (SELECT `eprint`.`eprin | 0.000    |
The other processes (queued) are all waiting for table level lock and look like this...

| 105134 | doras | localhost | doras | Query   | 487  | Waiting for table level lock | UPDATE `eprint` SET `edit_lock_user`='1037', `edit_lock_since`='1550139932', `edit_lock_until`='1550 | 0.000    |
| 105135 | doras | localhost | doras | Query   | 486  | Waiting for table level lock | SELECT `eprintid`,`rev_number`,`eprint_status`,`userid`,`importid`,`source`,`dir`,`datestamp_year`,` | 0.000

Eventually the max http processes is reached and the system crashes. The only remedy is to restart httpd and mariadb.

Any ideas what this insert into cache is and why it's locking the the system?
Thanks for any guidance
Fran Callaghan
Dublin City University


S?anadh R?omhphoist/Email Disclaimer

T? an r?omhphost seo agus aon chomhad a sheoltar leis faoi r?n agus is lena ?s?id ag an seola? agus sin amh?in ?. Is f?idir tuilleadh a l?amh anseo. <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww4.dcu.ie%2Fiss%2Fseanadh-riomhphoist.shtml&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cf0d6d6cec0d2498f630908d692911ac8%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=cxaNhEyQ4ARfTrV8n6pLkQNLFY4D5mNznKlyKqDbSR8%3D&amp;reserved=0>

This e-mail and any files transmitted with it are confidential and are intended solely for use by the addressee. Read more here.<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww4.dcu.ie%2Fiss%2Femail-disclaimer.shtml&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cf0d6d6cec0d2498f630908d692911ac8%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=gwxFEeE2ouXqh99%2FNiubE3Fel2bdIC1LlzRRvBQp1u4%3D&amp;reserved=0>





[Image removed by sender.]
[Image removed by sender.]<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2FDCU%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cf0d6d6cec0d2498f630908d692911ac8%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=SGpNapmgJ13Foz2bKoxkIJ2G9lXpaUZ8UAdYF8%2FGMOA%3D&amp;reserved=0>[Image removed by sender.]<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2FDublinCityUni&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cf0d6d6cec0d2498f630908d692911ac8%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=IucrkZ5QmuDb4FQ%2BRQwix2OQ7Qbl8qwcYZ82AHdTMFM%3D&amp;reserved=0>[Image removed by sender.]<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fdublin-city-university&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cf0d6d6cec0d2498f630908d692911ac8%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=o7kWkgfshghFiIcOeZ9WTbvvlZDQaKXJ6uyj7ycojLQ%3D&amp;reserved=0>[Image removed by sender.]<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.instagram.com%2Fdublincityuniversity%2F%3Fhl%3Den&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cf0d6d6cec0d2498f630908d692911ac8%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=KonMRDY%2BTJ3KrUbF7rOlZbRf6y%2F%2FcMiGQfl1wxS3tKc%3D&amp;reserved=0>[Image removed by sender.]<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fuser%2FDublinCityUniversity&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cf0d6d6cec0d2498f630908d692911ac8%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=w49RTSUafp2WGxHDMtmz%2F0adntw9ZwRtI35d%2BEdXhQk%3D&amp;reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/eprints-tech/attachments/20190214/d1456389/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ~WRD358.jpg
Type: image/jpeg
Size: 823 bytes
Desc: ~WRD358.jpg
Url : http://mailman.ecs.soton.ac.uk/pipermail/eprints-tech/attachments/20190214/d1456389/attachment-0001.jpg