[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[EP-tech] non-existent dates and oai-pmh sets



CAUTION: This e-mail originated outside the University of Southampton.
Thank you, John for that!  It worked for me, so I was able to validate/test correctly.

Also, thank you to David for the explanations about the date.
Based on what you wrote, since we don't have the "dates, dates, dates bazaar" plugin, and we are about to upgrade our repository to 3.4.3, so that should resolve this issue of 'invalid' dates being allowed into the repository.  What worries me is this part:
"If you had your own validation triggers set for individual date fields, then these would have stopped working without a second patch to fix this.  Both these patches can be found at the EPrints 3.4.3 release wiki page [1]."
I think we do make some checks on that (embargo period not exceeding a certain amount), so I need to test that.  If it is broken, I will try to install the patch from the release page.

Tomasz


________________________________________________
Tomasz Neugebauer
Senior Librarian | Biblioth?caire titulaire
Digital Projects & Systems Development Librarian / Biblioth?caire des Projets Num?riques & D?veloppement de Syst?mes
Concordia University / Universit? Concordia
Tel. / T?l. 514-848-2424 ext. / poste 7738
Email / courriel: tomasz.neugebauer at concordia.ca<mailto:tomasz.neugebauer at concordia.ca>
Mailing address / adresse postale: 1455 De Maisonneuve Blvd. W., LB-540-03, Montreal, Quebec H3G 1M8
Street address / adresse municipale: 1400 De Maisonneuve Blvd. W., LB-540-03, Montreal, Quebec H3G 1M8
library.concordia.ca

________________________________
From: John Salter <J.Salter at leeds.ac.uk>
Sent: Thursday, November 4, 2021 5:11 AM
To: eprints-tech at ecs.soton.ac.uk <eprints-tech at ecs.soton.ac.uk>; David R Newman <drn at ecs.soton.ac.uk>; Tomasz Neugebauer <Tomasz.Neugebauer at concordia.ca>
Subject: RE: [EP-tech] non-existent dates and oai-pmh sets


Attention This email originates from outside the concordia.ca domain. // Ce courriel provient de l'exterieur du domaine de concordia.ca



Hi Tomasz,

For the OAI-PMH question, you can add 'from' and 'until' parameters to the request - based on the last_mod date of the record in question e.g.:

https://[YOUR REPO]/cgi/oai2?verb=ListIdentifiers&metadataPrefix=oai_dc&set=[YOUR CUSTOM SET ID]&from=[A TIME JUST BEFORE THE LASTMOD]&until=[A TIME JUST AFTER THE LASTMOD]

The from/until params should be in the format: 2021-11-01T00:00:00Z



Automatic sets get listed on the item page* but it appears that the custom sets don't get listed.

I'll have to revisit the OAI-PMH specs to see if this is an allowed behaviour?



Cheers,

John





* e.g. EPrint 631 is part of our 'irus-orcid' set: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Feprints.whiterose.ac.uk%2Fcgi%2Foai2%3Fverb%3DListIdentifiers%26metadataPrefix%3Doai_dc%26set%3Dirus-orcid%26from%3D2021-11-01T00%3A00%3A00Z%26until%3D2021-11-02T00%3A00%3A00Z&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=11%2FwFIP6MSkzsRIK7kdQf02wcFBAh4WkT1h30H5w0Ho%3D&amp;reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Feprints.whiterose.ac.uk%2Fcgi%2Foai2%3Fverb%3DListIdentifiers%26metadataPrefix%3Doai_dc%26set%3Dirus-orcid%26from%3D2021-11-01T00%3A00%3A00Z%26until%3D2021-11-02T00%3A00%3A00Z&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=11%2FwFIP6MSkzsRIK7kdQf02wcFBAh4WkT1h30H5w0Ho%3D&amp;reserved=0>

but the irus-orcid set isn't listed on https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Feprints.whiterose.ac.uk%2Fcgi%2Foai2%3Fverb%3DGetRecord%26metadataPrefix%3Doai_dc%26identifier%3Doai%3Aeprints.whiterose.ac.uk%3A631&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=hr5W8poxX%2F71iMFcy%2FW2m45rQiy8%2FxQPsuVs7H1fC3I%3D&amp;reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Feprints.whiterose.ac.uk%2Fcgi%2Foai2%3Fverb%3DGetRecord%26metadataPrefix%3Doai_dc%26identifier%3Doai%3Aeprints.whiterose.ac.uk%3A631&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=hr5W8poxX%2F71iMFcy%2FW2m45rQiy8%2FxQPsuVs7H1fC3I%3D&amp;reserved=0>





From: eprints-tech-bounces at ecs.soton.ac.uk [mailto:eprints-tech-bounces at ecs.soton.ac.uk] On Behalf Of David R Newman via Eprints-tech
Sent: 03 November 2021 23:46
To: eprints-tech at ecs.soton.ac.uk; Tomasz Neugebauer <Tomasz.Neugebauer at concordia.ca>
Subject: Re: [EP-tech] non-existent dates and oai-pmh sets



Hi Tomasz,



Validation for dates has been added in EPrints 3.4.3, which prevent invalid dates being set.  Unfortunately, a couple of bugs have been found post release:



1. Multiple values not supported by EPrints::MetaField::Date validate function

2. Bespoke validations no longer supported for Date MetaField



Both of these issues would not be a problem in a vanilla EPrints archive but if you use the Dates, Dates, Dates Bazaar plugin, you will need to apply a patch to fix this.  If you had your own validation triggers set for individual date fields, then these would have stopped working without a second patch to fix this.  Both these patches can be found at the EPrints 3.4.3 release wiki page [1].



A further addition to the Date MetaField in 3.4.3 is the ability to define a bespoke function under $c->{worfklow_datepicker} that defines the HTML markup as a XML::LibXML::DocumentFragment.  This HTML markup could include client-side (i.e. JavaScript) validation/restrictions to prevent invalid dates from being set (e.g. if you change the month to February, its does not allow 30th or 31st to be set for the date).



Alternatively there is a Datepicker MetaField available in the daterangepicker ingredient [2].  However, this uses a single text field rather than multiple field separately storing year, month and day in separate database columns.  So it is not ideal if you want to transform an existing Date MetaField into a Datepicker MetaField.



On you second question.  There is no URL syntax to test if an item is in a particular OAI-PMH set.  I think that statement is true for all OAI-PMH not just specific to EPrints.  I am not sure there is any straightforward way to perform your test, bar writing your own client script to download just the identifiers for a whole OAI-PMH set, 100 identifiers at a time, as OAI-PMH is designed to work.  After downloading each tranche check whether the item in question is present.   The initial request for identifiers in a set would be a bit like:



https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fexample.eprints.org%2Fcgi%2Foai2%3Fverb%3DListIdentifiers%26metadataPrefix%3Doai_dc%26set%3D7374617475733D707562&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=%2BF0hLDUBLhferlv8GdTgyHCJcsx%2B0Tz5H%2F3cbwKTEs4%3D&amp;reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fexample.eprints.org%2Fcgi%2Foai2%3Fverb%3DListIdentifiers%26metadataPrefix%3Doai_dc%26set%3D7374617475733D707562&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=%2BF0hLDUBLhferlv8GdTgyHCJcsx%2B0Tz5H%2F3cbwKTEs4%3D&amp;reserved=0>



This will then give you a resumption token to get the next 100 until there are no more identifiers left.



Regards



David Newman



[1] https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.eprints.org%2Fw%2FEPrints_3.4.3%23Known_Issues&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=UdoNHOUwsal5yi%2BPlC0zzmx8DzvZkvZ4kltBMuNJsHo%3D&amp;reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.eprints.org%2Fw%2FEPrints_3.4.3%23Known_Issues&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=UdoNHOUwsal5yi%2BPlC0zzmx8DzvZkvZ4kltBMuNJsHo%3D&amp;reserved=0>

[2] https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feprints%2Fdaterangepicker%2Fblob%2Fmaster%2Fplugins%2FEPrints%2FMetaField%2FDatepicker.pm&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=mnDZO%2Fg9qVHspzUH7mknq87c1U32DfeY0i%2FP%2B%2B78Gew%3D&amp;reserved=0<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feprints%2Fdaterangepicker%2Fblob%2Fmaster%2Fplugins%2FEPrints%2FMetaField%2FDatepicker.pm&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630100733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=mnDZO%2Fg9qVHspzUH7mknq87c1U32DfeY0i%2FP%2B%2B78Gew%3D&amp;reserved=0>





On 03/11/2021 22:44, Tomasz Neugebauer via Eprints-tech wrote:

CAUTION: This e-mail originated outside the University of Southampton.

Hello everyone,



I have a couple of questions.



First question, it's about setting an embargo expiry date (and choosing other dates).  It was recently brought to my attention that it is possible to set "non-existent" dates, for example, February 31st or April 31st.  I'm not sure if I hadn't thought about this before, or if I didn't really think of this as an issue, but I suppose this raises the question of:

If the embargo expiry date (e.g: April 31st) never happens, then does the embargo ever expire?  ?   The question is, has anyone thought this issue serious enough to build-in some error checking for this, or attempted to modify the date-picker interface that exists in EPrints to address this issue?



Second question, it's about limiting what is inside an OAI-PMH set.  Based on my understanding, we can use something like this in the oai.pl configuration to create a set that is both (1) limited to un-embargoed (open access) items AND (2) items limited by type:



$oai->{custom_sets} = [

            {           spec => "openaire", name => "OpenAIRE Set - OA article conference book monograph",

                        filters=> [

                                    {meta_fields=>["full_text_status"], value=>"public"},

                                    {meta_fields =>[qw( type )], merge => "ANY", value => "article conference_item book_section monograph book" }

                        ]

            }

];

How can I confirm if this is working correctly?  In other words, how can I test if an item in the repository is part of an OAI-PMH set?  Is there a URL syntax where I could pass the eprintid and the set name to find out if it is part of the set?



Thanks!



Tomasz





________________________________________________

Tomasz Neugebauer
Senior Librarian | Biblioth?caire titulaire
Digital Projects & Systems Development Librarian / Biblioth?caire des Projets Num?riques & D?veloppement de Syst?mes
Concordia University / Universit? Concordia

Tel. / T?l. 514-848-2424 ext. / poste 7738
Email / courriel: tomasz.neugebauer at concordia.ca<mailto:tomasz.neugebauer at concordia.ca>

Mailing address / adresse postale: 1455 De Maisonneuve Blvd. W., LB-540-03, Montreal, Quebec H3G 1M8
Street address / adresse municipale: 1400 De Maisonneuve Blvd. W., LB-540-03, Montreal, Quebec H3G 1M8

library.concordia.ca



*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailman.ecs.soton.ac.uk%2Fmailman%2Flistinfo%2Feprints-tech&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630110692%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=fsNRGI7nIXT7Mk4SHxgJPVTJKytqfQfVXiZnnwHN5UE%3D&amp;reserved=0>

*** Archive: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630110692%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=UsEEYwyV%2BPxomAw2idIjRNs54jUTWt0tDk4z59b7%2BJA%3D&amp;reserved=0<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630110692%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=UsEEYwyV%2BPxomAw2idIjRNs54jUTWt0tDk4z59b7%2BJA%3D&amp;reserved=0>

*** EPrints community wiki: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630110692%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=mUnsqmlK1XdJPM1tnU0mC9yX25O39EPNL9nCuiUDtxw%3D&amp;reserved=0<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630110692%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=mUnsqmlK1XdJPM1tnU0mC9yX25O39EPNL9nCuiUDtxw%3D&amp;reserved=0>



[!]<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.avg.com%2Femail-signature%3Futm_medium%3Demail%26utm_source%3Dlink%26utm_campaign%3Dsig-email%26utm_content%3Demailclient&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630110692%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=yHlQ93E20P1joPOHrB90X6DRAtB%2BgPldgYrhUi1jlWM%3D&amp;reserved=0>

Virus-free. https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.avg.com%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630110692%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=nwETL7mR27HWHSv2OnQYbBEhcn%2BCIBmt47fs6xvl9jU%3D&amp;reserved=0<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.avg.com%2Femail-signature%3Futm_medium%3Demail%26utm_source%3Dlink%26utm_campaign%3Dsig-email%26utm_content%3Demailclient&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C733767dbe5794fb351cf08d9a2f08b0a%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637719977630110692%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=yHlQ93E20P1joPOHrB90X6DRAtB%2BgPldgYrhUi1jlWM%3D&amp;reserved=0>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/eprints-tech/attachments/20211108/845e67c4/attachment-0001.html