[EP-tech] Main/filename not set in staging area

[drn at ecs.soton.ac.uk (David R Newman)]

Hi James,

Having a look at File.pm and URL.pm in 3.4.4, the only change for 
File.pm since the initial 3.4 import that relates to how files upload is 
some code that strips out trailing and leading spaces before the file is 
saved to disk and its filename saved to the database.? There is no such 
change for URL.pm.

One thing that has been altered about file uploads to help deal with 
problematic filenames is in perl_lib/EPrints/Plugin/Storage/Local.pm.? 
This change allowed files to be renamed to <fileid>.bin on disk and 
EPrints would check this as a secondary filename if it could not find 
the file under its recorded filename in the database.? The mismatch may 
have been due to a special character being transposed differently in the 
database to the filesystem, the previous leading/trailing spaces issue 
or some other problem.? There is a setting ($c->{generic_filenames}) 
which is disabled by default but if enabled would save files to disk as 
<fileid>.bin in future rather than the original filename.? This was 
first implemented for EPrints 3.4.3.

None of what I describe above seems like it would lead to the specific 
issue you are reporting.? I am assuming that Staging.pm was based on 
File.pm at some point in the past.? If this diverged pre-3.4 then there 
may have been other changes between 3.3 and 3.4.

Regards

David Newman

On 08/12/2022 12:30, James Kerwin via Eprints-tech wrote:
> *CAUTION:* This e-mail originated outside the University of Southampton.
> Hi All,
>
> Once again I've done something to our poor Data Repository. We're on 
> EPrints 3.4.4.
>
> If we upload files the "normal" way they go on with no problem.
>
> If we do an upload of files over 4GB in our "Staging Area" then "Main" 
> is not set in the Documents database table. I'm almost certain that 
> this is something specific to my repository as we have some slightly 
> different files in:
>
> /usr/share/eprints/perl_lib/EPrints/Plugin/Screen/EPrint/UploadMethod
>
> compared to the github for EPrints 3.4.4:
>
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feprints%2Feprints3.4%2Ftree%2Fmaster%2Fperl_lib%2FEPrints%2FPlugin%2FScreen%2FEPrint%2FUploadMethod&amp;data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc44f0542842241f3463f08dad92521ff%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638061050739996773%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=mKDuGPAKdCjVHYsSAYLg9c%2FWbZM8cUFZu6a51Mba804%3D&amp;reserved=0 
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feprints%2Feprints3.4%2Ftree%2Fmaster%2Fperl_lib%2FEPrints%2FPlugin%2FScreen%2FEPrint%2FUploadMethod&amp;data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc44f0542842241f3463f08dad92521ff%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638061050739996773%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=mKDuGPAKdCjVHYsSAYLg9c%2FWbZM8cUFZu6a51Mba804%3D&amp;reserved=0>
>
> We have File.pm, URL.pm and Staging.pm. Some changes have been made to 
> these files to enable the staging area. I've no idea how customised 
> this is compared to other repositories. Other values appear to be 
> getting set and the "filename" is being set in the Files table.
>
> Before I go through a semi-destructive process of trying to set this 
> value, is there something very obvious that I'm missing?
>
> Thanks,
> James
>
> *** Options:http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc44f0542842241f3463f08dad92521ff%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638061050739996773%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=tXjjiaQ93xo69On0Ybo5LCfZC2eWbP8RZyQFihDLIFM%3D&amp;reserved=0
> *** EPrints community wiki:https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=05%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7Cc44f0542842241f3463f08dad92521ff%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C638061050740152995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=MTP385AevaFoxG07jvp%2BmBJ8x%2FHQzViRqYwMSAKbzg4%3D&amp;reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/eprints-tech/attachments/20221208/0db790dc/attachment-0001.html