[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[EP-tech] IRstats2 "origin of downloads"
Hi Tomasz,
In fact, I think I found an issue even with that amended code in one
scenario, so in a development version of IRStats2 are changed that to:
? ? foreach my $pkg ( 'Geo::IP', 'Geo::IP::PurePerl' )
?? ?{
?? ???? if( EPrints::Utils::require_if_exists( $pkg ) )
?? ???? {
?? ???? ??? if( $pkg !~ /PurePerl/ )
?? ???? ??? {
?? ???? ??? ??? $self->{geoip} = $pkg->new( $dat_file ) if $dat_file eq
'1';
?? ???? ??? ??? $self->{geoip} = $pkg->open( $dat_file ) if $dat_file
ne '1';
?? ???? ??? }
?? ???? ??? else
?? ???? ??? {
?? ???? ??? ??? $self->{geoip} = $pkg->new( $dat_file );
?? ???? ??? }
?? ???? ??? last if( defined $self->{geoip} );
?? ???? }
?? ?}
Regards
David Newman
On 20/04/2023 11:23 pm, David R Newman via Eprints-tech wrote:
> Hi Tomasz,
>
> Ah, now you say this that rings a bell.? We have been looking at new
> release of IRstats2 and remember that there was a (pretty old) change
> since the last release that that uses ->new for one of the GeoIP
> libraries and ->open for the other:
>
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feprints%2Firstats2%2Fcommit%2Fa84f22bf6d8a7faa9b6593afa97ef1a0cd360fcc&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C99f15a4056524e4d22b908db423a6393%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176590786071565%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=onw05culW80jEJ2pWwGrUEOlDJ9I6f9L2j%2F3Ejxddz0%3D&reserved=0
>
> Regards
>
> David Newman
>
>
> On 20/04/2023 11:03 pm, Tomasz Neugebauer wrote:
>> *CAUTION:* This e-mail originated outside the University of Southampton.
>>
>> Hm? I think that one thing that might have happened here is that when
>> I upgraded to 3.4, I removed the GeoIP.dat file from the ?lib?
>> folder, thinking that the library is supposed to use ?the global one?
>> instead.? I?m talking about this part of the code:
>>
>> my $dat_file = $self->{session}->config( "lib_path").'/geoip/GeoIP.dat';
>>
>> # alternatively use the global one
>> $dat_file = 1 if( !-e $dat_file );
>>
>> However, now that I look at this code, I find myself really
>> confused!? $dat_file is set to 1 if the path doesn?t exist?! ?Let?s
>> just say that doesn?t look solid to me. What is setting that path to
>> 1 supposed to accomplish? The loop later on, I needed to use the
>> ?open? command if I specify a path, not the ?new? command here:
>>
>> $self->{geoip} = $pkg->open( $dat_file );
>>
>> However, if that path in lib doesn?t exist, I?m opening ?1?, whatever
>> that means ? so that fails with an error; ?new? works on it, but not
>> sure what it does, then it defaults to the global one? I think that?s
>> where possibly things are just failing.? When the file is there in
>> the lib folder, it needs to be ?open? command, not new, but when the
>> file is not there, that?s when it?s ?new? with that magic number ?1??
>>
>> Tomasz
>>
>> *From:*David R Newman <drn at ecs.soton.ac.uk>
>> *Sent:* April 19, 2023 3:23 PM
>> *To:* Tomasz Neugebauer <Tomasz.Neugebauer at concordia.ca>;
>> eprints-tech at ecs.soton.ac.uk
>> *Subject:* Re: [EP-tech] IRstats2 "origin of downloads"
>>
>> *Attention* This email originates from outside the concordia.ca
>> domain. // Ce courriel provient de l'ext?rieur du domaine de concordia.ca
>>
>> Hi Tomasz,
>>
>> Another possibility is that the irstats2_countries table has crashed
>> an is therefore no longer updated.? However, I think you would have
>> likely noticed this.? As I suspect it would have prevented other
>> parts of IRStats2 updating, as one day the irstats2_internal table
>> will retain the flag saying stats are still being updated and then
>> subsequent days the process_stats would not be able to run.? However,
>> maybe a crash table does not cause process_stats to fail without
>> clearing the update flag.
>>
>> Otherwise, there is various checks in the code that will stop country
>> or origin data being generated.? Here is the whole block of code.?
>> Hopefully, you can work out from this whether something on your
>> repository server does not meet the required criteria:
>>
>> # if possible, use the GeoIP data file shipped with EPrints
>> ?? ?my $dat_file = $self->{session}->config(
>> "lib_path").'/geoip/GeoIP.dat';
>>
>> ?? ?# alternatively use the global one
>> ?? ?$dat_file = 1 if( !-e $dat_file );
>>
>> ?? ?#Test Geo::IP first - it's faster!
>> ?? ?foreach my $pkg ( 'Geo::IP', 'Geo::IP::PurePerl' )
>> ?? ?{
>> ?? ???? if( EPrints::Utils::require_if_exists( $pkg ) )
>> ?? ???? {
>> ?? ???? ??? $self->{geoip} = $pkg->new( $dat_file );
>> ?? ???? ??? last if( defined $self->{geoip} );
>> ?? ???? }
>> ?? ?}
>>
>> ?? ?if( !defined $self->{geoip} )
>> ?? ?{
>> ?? ???? $self->{advertise} = 0;
>> ?? ???? $self->{disable} = 1;
>> ?? ???? $self->{error} = "Failed to load required module for
>> Processor::Access::Country. Country information will not be available.";
>> ?? ???? return $self;
>> ?? ?}
>>
>> This code is in the file
>> EPRINS_PATH/plugins/EPrints/Plugin/Stats/Processor/Access/Country.pm
>> so if you cannot work out whether the criteria is meet, you could try
>> adding some debug and then running process_stats (or wait for it to
>> run overnight).? If this was failing I would hope that wherever you
>> are logging the output (STDOUT and STDERR) fro process_stats would
>> show the $self->{error} message above.
>>
>> Regards
>>
>> David Newman
>>
>> On 19/04/2023 5:33 pm, Tomasz Neugebauer wrote:
>>
>> *CAUTION:*This e-mail originated outside the University of
>> Southampton.
>>
>> Hi David,
>>
>> Thanks for that, but I do believe that we have those packages
>> installed. We are running on Ubuntu 18.04 if that makes a difference.
>>
>> Here is some output from apt:
>>
>> ?apt list | grep geoip-database
>>
>> geoip-database/bionic,now 20180315-1 all [installed]
>>
>> geoip-database-contrib/bionic 1.19 all
>>
>> geoip-database-extra/bionic,now 20180315-1 all
>> [installed,automatic]
>>
>> ?apt list | grep libgeo-ip-perl
>>
>> ??????libgeo-ip-perl/bionic,now 1.51-1 amd64 [installed]
>>
>> Tomasz
>>
>> ------------------------------------------------------------------------
>>
>> *From:*David R Newman <drn at ecs.soton.ac.uk>
>> <mailto:drn at ecs.soton.ac.uk>
>> *Sent:* Wednesday, April 19, 2023 5:42 AM
>> *To:* eprints-tech at ecs.soton.ac.uk <eprints-tech at ecs.soton.ac.uk>
>> <mailto:eprints-tech at ecs.soton.ac.uk>; Tomasz Neugebauer
>> <Tomasz.Neugebauer at concordia.ca>
>> <mailto:Tomasz.Neugebauer at concordia.ca>
>> *Subject:* Re: [EP-tech] IRstats2 "origin of downloads"
>>
>> *Attention*This email originates from outside the concordia.ca
>> domain. // Ce courriel provient de l'ext?rieur du domaine de
>> concordia.ca
>>
>> Hi Tomasz,
>>
>> You will need an operating system package and a Perl library
>> (which can probably be installed as an OS package) for that to
>> work.? On Ubuntu 20.04 that is:
>>
>> apt install geoip-database libgeo-ip-perl
>>
>> On Enterprise Linux (RHEL, CentOS, Fedora, Rocky Linux, Alma
>> Linux, etc.) it is:
>>
>> yum install GeoIP perl-Geo-IP
>>
>> I do not think you will be able to get back missing country of
>> origin data without regenerating your stats from scratch but this
>> should start being added in future.
>>
>> Regards
>>
>> David Newman
>>
>> On 18/04/2023 21:29, Tomasz Neugebauer via Eprints-tech wrote:
>>
>> *CAUTION:*This e-mail originated outside the University of
>> Southampton.
>>
>> I?m trying to figure out why the ?origin of downloads?
>> country data has stopped working for our repository.
>>
>> I see much of the old (pre-2022) data is still there, but
>> starting in 2022, ?origin of downloads? (the map
>> visualization) contains no data.
>>
>> Any idea as to what could be causing that or where to look
>> for the issue?
>>
>> It?s not just the visualization of the map, because that
>> still works for ?all? and earlier date ranges.
>>
>> It looks like the country data is not there starting in 2022?
>>
>> Tomasz
>>
>>
>>
>>
>> *** Options:
>> http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech <http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech>
>>
>> *** Archive:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C99f15a4056524e4d22b908db423a6393%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176590786071565%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=vlH9TZTiO2h6E72Q3PQa6gRpawM9vTO4dOVMOvObZNg%3D&reserved=0 <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C99f15a4056524e4d22b908db423a6393%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176590786071565%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=vlH9TZTiO2h6E72Q3PQa6gRpawM9vTO4dOVMOvObZNg%3D&reserved=0>
>>
>> *** EPrints community wiki:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C99f15a4056524e4d22b908db423a6393%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176590786071565%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hIzYzxYsURmOfSKc3k4%2BGgZSq8t3AZaFfkm8vvgtM6s%3D&reserved=0 <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C99f15a4056524e4d22b908db423a6393%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176590786071565%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hIzYzxYsURmOfSKc3k4%2BGgZSq8t3AZaFfkm8vvgtM6s%3D&reserved=0>
>>
>
>
> *** Options:http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C99f15a4056524e4d22b908db423a6393%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176590786071565%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=vlH9TZTiO2h6E72Q3PQa6gRpawM9vTO4dOVMOvObZNg%3D&reserved=0
> *** EPrints community wiki:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C99f15a4056524e4d22b908db423a6393%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176590786071565%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hIzYzxYsURmOfSKc3k4%2BGgZSq8t3AZaFfkm8vvgtM6s%3D&reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/eprints-tech/attachments/20230421/23b35c0c/attachment-0001.html