[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[EP-tech] Error when running irstats



Hi,

Previously, thank you for your help. The process of the process_stats
command is complete.

I also read your e-mail, and I want to ask, does this mean we have to make
changes from utf8 to utfmb4? I read the process of moving from utf8 to
utf8mb4 is quite complicated, and the fear has an impact on the database.

When the process_stats command is complete, a description appears as
follows:

Access: it took 152588 secs to parse 8741665 records (average = 57.29
records / sec)
Access: 3533033 records kept out of 8741665 (ratio = 40.42%)

[image: image.png]

In the line above, does that mean there are around 3533033 records that are
not entered?

Thank you,

Best regards,
Agung Prasetyo W.

On Thu, Feb 14, 2019 at 10:58 PM John Salter via Eprints-tech <
eprints-tech at ecs.soton.ac.uk> wrote:

> Hi Agung,
> I think this might be a problem with how MySQL handles UTF-8 data.
>
> What MySQL refers to as UTF-8 cannot handle 4-byte UTF-8 characters.
> Yes - this is very stupid
>
> This explains it well:
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmathiasbynens.be%2Fnotes%2Fmysql-utf8mb4&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=TDnsCYsQxBn5Ol3MBu2gee7ZnREJ5qteLforb3pb6Jc%3D&amp;reserved=0,
> and has some solutions.
>
> This might also be useful:
>
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feprints%2Feprints%2Fissues%2F386&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=izi%2BNYknjYnnzp38zOdtpVe5PU86705DOmIQuitQ0mg%3D&amp;reserved=0
>
> Does that help at all?
>
> Cheers,
> John
>
> -----Original Message-----
> From: eprints-tech-bounces at ecs.soton.ac.uk [mailto:
> eprints-tech-bounces at ecs.soton.ac.uk] On Behalf Of Yuri Carrer via
> Eprints-tech
> Sent: 13 February 2019 08:04
> To: eprints-tech at ecs.soton.ac.uk
> Subject: Re: [EP-tech] Error when running irstats
>
> in /usr/share/eprints3/lib/plugins/EPrints/Stats/Handler.pm , at the
> beginning.
>
> Il 13/02/19 08:36, Ajunk Pracetio ha scritto:
> > Hi,
> >
> > Where I can activate this debug?
> >
> > Thank you
> >
> > Best regards,
> > Agung Prasetyo W.
> >
> > On Wed, Feb 13, 2019 at 2:16 PM Yuri via Eprints-tech
> > <eprints-tech at ecs.soton.ac.uk <mailto:eprints-tech at ecs.soton.ac.uk>>
> > wrote:
> >
> >
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F10957238%2Fincorrect-string-value-when-trying-to-insert-utf-8-into-mysql-via-jdbc&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=1W3BnrDBXl7%2F0AyC25Ah0X8cOqxov%2B%2BIND0MB2FQBvQ%3D&amp;reserved=0
> >
> >     maybe this?
> >
> >     \xF76 should be a four byte utf-8. But Eprints itself don't have this
> >     problem, so it is something in Irstats connection or irstat tables
> >     with
> >     different column encoding. Anyway, you can activate
> >
> >     my $DEBUG_SQL = 0; (switch to 1) to debug the sql.
> >
> >     Il 13/02/19 01:58, Ajunk Pracetio via Eprints-tech ha scritto:
> >     > Hi,
> >     >
> >     > Previously, I migrated eprints 3.3.12 to 3.4 on Ubuntu 12.04 LTS.
> >     > Currently I have managed to move eprints from an old server to a
> >     > server using Ubuntu 16.04 LTS.
> >     >
> >     > Only, when doing the irstats process, the *process_stats REPOID
> >     > --setup --verbose* command appears an error like the following:
> >     >
> >     > *DBD::mysql::st execute failed: Incorrect string value: '\xF76' for
> >     > column 'value' at row 1 at
> >     > /usr/share/eprints3/lib/plugins/EPrints/Stats/Handler.pm line 653.
> >     > ....
> >     > DBD::mysql::st execute failed: Data too long for column 'value'
> >     at row
> >     > 1 at /usr/share/eprints3/lib/plugins/EPrints/Stats/Handler.pm
> >     line 653.*
> >     >
> >     > Did this error occur because our previous database actually came
> >     from
> >     > the version of mysql 5.5.54 (eprints 3.3.12 and Ubuntu 12.04) and
> >     > changed to mysql 5.7.25 (eprints 3.4 and Ubuntu 16.04).
> >     >
> >     > I have searched Google and haven't found a solution. Can anyone
> >     help
> >     > us with this error? Then will this error affect the amount of data
> >     > displayed on IRStats?
> >     >
> >     > thank you.
> >     >
> >     > Best regards.
> >     > Agung Prasetyo W.
> >     >
> >     > --
> >     > Life isn't valuable if you don't have someone to care
> >     >
> >     > Blog :
> >
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fagungprasetyo.net%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=MTi52Yulx7W7X%2FRlw%2Fb60hRIZ%2BbGY5XPQtkv3y8fyRc%3D&amp;reserved=0
> >
> >     >
> >     <
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fagungprasetyo.net%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=MTi52Yulx7W7X%2FRlw%2Fb60hRIZ%2BbGY5XPQtkv3y8fyRc%3D&amp;reserved=0
> >
> >     > IG & Twitter : prazetyo
> >     >
> >     > *** Options:
> >     http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> >     > *** Archive:
> >
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=z%2FsAAveF3XPR2q92u2rz32bD6EEvONxbthuRXx%2FozDM%3D&amp;reserved=0
> >     > *** EPrints community wiki:
> >
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=lcTyjpIfN5kFBCkkRdu0QX7%2BA0yoFSpI93CPrmbCdq8%3D&amp;reserved=0
> >     > *** EPrints developers Forum:
> >
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fforum.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=enxLs%2Fcn4g9f95DSzaPydiXH41SogdAUUXYe91FPL4Q%3D&amp;reserved=0
> >     *** Options:
> >     http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> >     *** Archive:
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=z%2FsAAveF3XPR2q92u2rz32bD6EEvONxbthuRXx%2FozDM%3D&amp;reserved=0
> >     *** EPrints community wiki:
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=lcTyjpIfN5kFBCkkRdu0QX7%2BA0yoFSpI93CPrmbCdq8%3D&amp;reserved=0
> >     *** EPrints developers Forum:
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fforum.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=enxLs%2Fcn4g9f95DSzaPydiXH41SogdAUUXYe91FPL4Q%3D&amp;reserved=0
> >
> >
> >
> > --
> > Life isn't valuable if you don't have someone to care
> >
> > Blog :
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fagungprasetyo.net%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=MTi52Yulx7W7X%2FRlw%2Fb60hRIZ%2BbGY5XPQtkv3y8fyRc%3D&amp;reserved=0
> > IG & Twitter : prazetyo
>
> --
> Yuri Carrer
>
>   CAB - Centro di Ateneo per le Biblioteche, Universit? di Padova
>   Tel: 049/827 3615 - Via Anghinoni, 3 - Padova
>
>
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive:
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=z%2FsAAveF3XPR2q92u2rz32bD6EEvONxbthuRXx%2FozDM%3D&amp;reserved=0
> *** EPrints community wiki:
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=lcTyjpIfN5kFBCkkRdu0QX7%2BA0yoFSpI93CPrmbCdq8%3D&amp;reserved=0
> *** EPrints developers Forum:
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fforum.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=enxLs%2Fcn4g9f95DSzaPydiXH41SogdAUUXYe91FPL4Q%3D&amp;reserved=0
>
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=z%2FsAAveF3XPR2q92u2rz32bD6EEvONxbthuRXx%2FozDM%3D&amp;reserved=0
> *** EPrints community wiki: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=lcTyjpIfN5kFBCkkRdu0QX7%2BA0yoFSpI93CPrmbCdq8%3D&amp;reserved=0
> *** EPrints developers Forum: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fforum.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=enxLs%2Fcn4g9f95DSzaPydiXH41SogdAUUXYe91FPL4Q%3D&amp;reserved=0
>


-- 
Life isn't valuable if you don't have someone to care

Blog : https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fagungprasetyo.net%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc2a83ae34ebf4f2562df08d692e3cbe0%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&amp;sdata=MTi52Yulx7W7X%2FRlw%2Fb60hRIZ%2BbGY5XPQtkv3y8fyRc%3D&amp;reserved=0
IG & Twitter : prazetyo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/eprints-tech/attachments/20190215/db647c3d/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 4750 bytes
Desc: not available
Url : http://mailman.ecs.soton.ac.uk/pipermail/eprints-tech/attachments/20190215/db647c3d/attachment-0001.png