[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[EP-tech] IRstats2 "origin of downloads"
Hi Tomasz,
Ah, now you say this that rings a bell.? We have been looking at new
release of IRstats2 and remember that there was a (pretty old) change
since the last release that that uses ->new for one of the GeoIP
libraries and ->open for the other:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feprints%2Firstats2%2Fcommit%2Fa84f22bf6d8a7faa9b6593afa97ef1a0cd360fcc&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C30a78478048043e1aa0108db41edd8c1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176262035957387%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=BxYVDp90j%2FrAY30yy9OfZn2cP5FRONawOLEsdt1iXb0%3D&reserved=0
Regards
David Newman
On 20/04/2023 11:03 pm, Tomasz Neugebauer wrote:
> *CAUTION:* This e-mail originated outside the University of Southampton.
>
> Hm? I think that one thing that might have happened here is that when
> I upgraded to 3.4, I removed the GeoIP.dat file from the ?lib? folder,
> thinking that the library is supposed to use ?the global one?
> instead.? I?m talking about this part of the code:
>
> my $dat_file = $self->{session}->config( "lib_path").'/geoip/GeoIP.dat';
>
> # alternatively use the global one
> $dat_file = 1 if( !-e $dat_file );
>
> However, now that I look at this code, I find myself really confused!?
> $dat_file is set to 1 if the path doesn?t exist?! ?Let?s just say that
> doesn?t look solid to me. What is setting that path to 1 supposed to
> accomplish? The loop later on, I needed to use the ?open? command if I
> specify a path, not the ?new? command here:
>
> $self->{geoip} = $pkg->open( $dat_file );
>
> However, if that path in lib doesn?t exist, I?m opening ?1?, whatever
> that means ? so that fails with an error; ?new? works on it, but not
> sure what it does, then it defaults to the global one? I think that?s
> where possibly things are just failing.? When the file is there in the
> lib folder, it needs to be ?open? command, not new, but when the file
> is not there, that?s when it?s ?new? with that magic number ?1??
>
> Tomasz
>
> *From:*David R Newman <drn at ecs.soton.ac.uk>
> *Sent:* April 19, 2023 3:23 PM
> *To:* Tomasz Neugebauer <Tomasz.Neugebauer at concordia.ca>;
> eprints-tech at ecs.soton.ac.uk
> *Subject:* Re: [EP-tech] IRstats2 "origin of downloads"
>
> *Attention* This email originates from outside the concordia.ca
> domain. // Ce courriel provient de l'ext?rieur du domaine de concordia.ca
>
> Hi Tomasz,
>
> Another possibility is that the irstats2_countries table has crashed
> an is therefore no longer updated.? However, I think you would have
> likely noticed this.? As I suspect it would have prevented other parts
> of IRStats2 updating, as one day the irstats2_internal table will
> retain the flag saying stats are still being updated and then
> subsequent days the process_stats would not be able to run.? However,
> maybe a crash table does not cause process_stats to fail without
> clearing the update flag.
>
> Otherwise, there is various checks in the code that will stop country
> or origin data being generated.? Here is the whole block of code.?
> Hopefully, you can work out from this whether something on your
> repository server does not meet the required criteria:
>
> # if possible, use the GeoIP data file shipped with EPrints
> ?? ?my $dat_file = $self->{session}->config(
> "lib_path").'/geoip/GeoIP.dat';
>
> ?? ?# alternatively use the global one
> ?? ?$dat_file = 1 if( !-e $dat_file );
>
> ?? ?#Test Geo::IP first - it's faster!
> ?? ?foreach my $pkg ( 'Geo::IP', 'Geo::IP::PurePerl' )
> ?? ?{
> ?? ???? if( EPrints::Utils::require_if_exists( $pkg ) )
> ?? ???? {
> ?? ???? ??? $self->{geoip} = $pkg->new( $dat_file );
> ?? ???? ??? last if( defined $self->{geoip} );
> ?? ???? }
> ?? ?}
>
> ?? ?if( !defined $self->{geoip} )
> ?? ?{
> ?? ???? $self->{advertise} = 0;
> ?? ???? $self->{disable} = 1;
> ?? ???? $self->{error} = "Failed to load required module for
> Processor::Access::Country. Country information will not be available.";
> ?? ???? return $self;
> ?? ?}
>
> This code is in the file
> EPRINS_PATH/plugins/EPrints/Plugin/Stats/Processor/Access/Country.pm
> so if you cannot work out whether the criteria is meet, you could try
> adding some debug and then running process_stats (or wait for it to
> run overnight).? If this was failing I would hope that wherever you
> are logging the output (STDOUT and STDERR) fro process_stats would
> show the $self->{error} message above.
>
> Regards
>
> David Newman
>
> On 19/04/2023 5:33 pm, Tomasz Neugebauer wrote:
>
> *CAUTION:*This e-mail originated outside the University of
> Southampton.
>
> Hi David,
>
> Thanks for that, but I do believe that we have those packages
> installed. We are running on Ubuntu 18.04 if that makes a difference.
>
> Here is some output from apt:
>
> ?apt list | grep geoip-database
>
> geoip-database/bionic,now 20180315-1 all [installed]
>
> geoip-database-contrib/bionic 1.19 all
>
> geoip-database-extra/bionic,now 20180315-1 all
> [installed,automatic]
>
> ?apt list | grep libgeo-ip-perl
>
> ??????libgeo-ip-perl/bionic,now 1.51-1 amd64 [installed]
>
> Tomasz
>
> ------------------------------------------------------------------------
>
> *From:*David R Newman <drn at ecs.soton.ac.uk>
> <mailto:drn at ecs.soton.ac.uk>
> *Sent:* Wednesday, April 19, 2023 5:42 AM
> *To:* eprints-tech at ecs.soton.ac.uk <eprints-tech at ecs.soton.ac.uk>
> <mailto:eprints-tech at ecs.soton.ac.uk>; Tomasz Neugebauer
> <Tomasz.Neugebauer at concordia.ca>
> <mailto:Tomasz.Neugebauer at concordia.ca>
> *Subject:* Re: [EP-tech] IRstats2 "origin of downloads"
>
> *Attention*This email originates from outside the concordia.ca
> domain. // Ce courriel provient de l'ext?rieur du domaine de
> concordia.ca
>
> Hi Tomasz,
>
> You will need an operating system package and a Perl library
> (which can probably be installed as an OS package) for that to
> work.? On Ubuntu 20.04 that is:
>
> apt install geoip-database libgeo-ip-perl
>
> On Enterprise Linux (RHEL, CentOS, Fedora, Rocky Linux, Alma
> Linux, etc.) it is:
>
> yum install GeoIP perl-Geo-IP
>
> I do not think you will be able to get back missing country of
> origin data without regenerating your stats from scratch but this
> should start being added in future.
>
> Regards
>
> David Newman
>
> On 18/04/2023 21:29, Tomasz Neugebauer via Eprints-tech wrote:
>
> *CAUTION:*This e-mail originated outside the University of
> Southampton.
>
> I?m trying to figure out why the ?origin of downloads? country
> data has stopped working for our repository.
>
> I see much of the old (pre-2022) data is still there, but
> starting in 2022, ?origin of downloads? (the map
> visualization) contains no data.
>
> Any idea as to what could be causing that or where to look for
> the issue?
>
> It?s not just the visualization of the map, because that still
> works for ?all? and earlier date ranges.
>
> It looks like the country data is not there starting in 2022?
>
> Tomasz
>
>
>
>
> *** Options:
> http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech <http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech>
>
> *** Archive:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C30a78478048043e1aa0108db41edd8c1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176262035957387%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=n380eHwXr17ds3TjclVkks229q95s1WJdlXMNcF1qIQ%3D&reserved=0 <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C30a78478048043e1aa0108db41edd8c1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176262035957387%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=n380eHwXr17ds3TjclVkks229q95s1WJdlXMNcF1qIQ%3D&reserved=0>
>
> *** EPrints community wiki:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C30a78478048043e1aa0108db41edd8c1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176262035957387%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=w8DOLl8ffaGA2vt0hXDMn9D8YFOODFIdD2daAL%2FU30Q%3D&reserved=0 <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C30a78478048043e1aa0108db41edd8c1%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638176262035957387%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=w8DOLl8ffaGA2vt0hXDMn9D8YFOODFIdD2daAL%2FU30Q%3D&reserved=0>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/eprints-tech/attachments/20230420/8ca10091/attachment-0001.html