[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[EP-tech] Re: Importing files from another system
That's fine with me :o)
I've been trying to get my head round the new codebase (I'm used to v<3.2. Now looking at 3.3), and trying to work out if the filename could get set automatically.
Running the import tool without a filename is broken.
It looks like all other bits of data work OK.
Below is the output when just a ...<file><url>file:///...</url></file>... is used.
I think this could be fixed by adding a test to Eprint::DataObj::File::create_from_data that checks that
$data->{filename} exists before using it. If it doesn't exist, extract it from $filepath (I'm guessing there may already be a function somewhere in EPrints that would do this)?
Cheers,
John
$ more VERSION
eprints-3.3.7
EPrints 3.3.7 (Bilberry) [Born on 2011-12-09]
$ bin/import digitallibrary --verbose --enable-file-imports --enable-web-imports --user 1 archive XML /tmp/files/test.xml
Starting EPrints Repository.
Connecting to DB ... done.
Use of uninitialized value in string eq at /opt/eprints3/bin/../perl_lib/EPrints/DataObj/Document.pm line 308.
Use of uninitialized value in string eq at /opt/eprints3/bin/../perl_lib/EPrints/DataObj/Document.pm line 308.
Use of uninitialized value $filename in pattern match (m//) at (eval 440) line 145.
Use of uninitialized value $filename in pattern match (m//) at (eval 440) line 15.
Use of uninitialized value $filename in pattern match (m//) at (eval 440) line 19.
Use of uninitialized value $filename in pattern match (m//) at (eval 440) line 23.
Use of uninitialized value $filename in pattern match (m//) at (eval 440) line 27.
Use of uninitialized value $filename in substitution (s///) at /opt/eprints3/perl_lib/EPrints/Plugin/Storage/Local.pm line 249.
Use of uninitialized value $filename in substitution (s///) at /opt/eprints3/perl_lib/EPrints/Plugin/Storage/Local.pm line 250.
Use of uninitialized value $filename in substitution (s///) at /opt/eprints3/perl_lib/EPrints/Plugin/Storage/Local.pm line 251.
Use of uninitialized value $fn in concatenation (.) or string at /opt/eprints3/perl_lib/EPrints/Plugin/Storage/Local.pm line 49.
Use of uninitialized value $fn in concatenation (.) or string at /opt/eprints3/perl_lib/EPrints/Plugin/Storage/Local.pm line 55.
Use of uninitialized value $fn in concatenation (.) or string at /opt/eprints3/perl_lib/EPrints/Plugin/Storage/Local.pm line 57.
Unable to write to /opt/eprints3/archives/digitallibrary/documents/disk0/00/00/00/42/01/: Is a directory
EPrints::Storage: No plugins available to store file/361
Use of uninitialized value in concatenation (.) or string at /opt/eprints3/bin/../perl_lib/EPrints/DataObj/File.pm line 764.
file/361::set_file() failed: No storage plugins succeeded
Failed to create subdataobj on document.files at /opt/eprints3/bin/../perl_lib/EPrints/DataSet.pm line 1008
Failed to create subdataobj on archive.documents at /opt/eprints3/bin/../perl_lib/EPrints/DataSet.pm line 1008
Ending EPrints Repository.
-----Original Message-----
From: eprints-tech-bounces at ecs.soton.ac.uk [mailto:eprints-tech-bounces at ecs.soton.ac.uk] On Behalf Of sf2
Sent: 21 June 2012 19:34
To: eprints-tech at ecs.soton.ac.uk
Subject: [EP-tech] Re: Importing files from another system
Hi John,
You can safely ignore all the "id" fields as the import script will
override them. I think you're right in saying that EPrints will set the
mime type by itself. I'm not too sure whether you need to include the
"main" and "filename" :-/
I've added your XML to the wiki page if that's ok with you
(http://wiki.eprints.org/w/Import_Files)!
Seb.
On Thu, 21 Jun 2012 18:07:13 +0100, John Salter <J.Salter at leeds.ac.uk>
wrote:
> Hi Seb,
> Thanks for this - but how much of the data structure should I include
when
> trying to make a new eprint?
> E.g. I don't have an eprintid, docid, fileid, so can't include them.
>
> I /could/ supply a mime_type, but should eprints be able to work this out
> from the file?
> Should the filename be calculated from the <url>, or do I have to specify
> this? (I see some fun with XSLT coming my way :o)
>
>>From a bit more testing, I think this is the minimal set of data to get a
>>file into the system, from the local file system (although you'd want to
>>map more data in there if you had it!):
>
> <?xml version='1.0' encoding='utf-8'?>
> <eprints xmlns='http://eprints.org/ep2/data/2.0'>
> <eprint>
> <documents>
> <document>
> <files>
> <file>
> <datasetid>document</datasetid>
> <filename>thumb.jpg</filename>
> <url>file:///tmp/files/thumb.jpg</url>
> </file>
> </files>
> <format>image</format>
> <security>public</security>
> <main>thumb.jpg</main>
> </document>
> </documents>
> <eprint_status>archive</eprint_status>
> <userid>1</userid>
> <type>image</type>
> <creators>
> <item>
> <name>
> <family>test</family>
> <given>test</given>
> </name>
> <id>test at test.com</id>
> </item>
> </creators>
> <title>test</title>
> </eprint>
> </eprints>
>
> Hope that helps someone in the future - and thanks for your help today!
> John
>
> -----Original Message-----
> From: eprints-tech-bounces at ecs.soton.ac.uk
> [mailto:eprints-tech-bounces at ecs.soton.ac.uk] On Behalf Of Sebastien
> Francois
> Sent: 21 June 2012 15:08
> To: eprints-tech at ecs.soton.ac.uk
> Subject: [EP-tech] Re: Importing files from another system
>
> Hi John,
>
> See an example attached - tested on 3.3.9, imported this way:
>
> bin/import <REPO_ID> --enable-file-imports --enable-web-imports --user 1
> --verbose eprint XML ./export2.xml
>
> Seb.
>
> PS: in the attached XML, I replaced the URL of my dev machine with
> __REPO_URL__, change this to your repository URL.
>
> On 21/06/12 14:55, John Salter wrote:
>> Yep - that didn't work either :o\
>>
>> According to this:
>> http://wiki.eprints.org/w/Import_From_URL
>> I was also missing a<files> around the<file>.
>> I've added that and re-tried all the options, so far none work :o\
>>
>> Any other suggestions?
>>
>> Cheers,
>> John
>>
>> -----Original Message-----
>> From: eprints-tech-bounces at ecs.soton.ac.uk
>> [mailto:eprints-tech-bounces at ecs.soton.ac.uk] On Behalf Of Jose Martin
>> Sent: 21 June 2012 14:40
>> To: eprints-tech at ecs.soton.ac.uk
>> Subject: [EP-tech] Re: Importing files from another system
>>
>> Have you tried for local files
>>
>> <url>file:///path_to_file</url>
>>
>> ?
>>
>> -----Original Message-----
>> From: eprints-tech-bounces at ecs.soton.ac.uk
>> [mailto:eprints-tech-bounces at ecs.soton.ac.uk] On Behalf Of John Salter
>> Sent: 21 June 2012 14:24
>> To: 'eprints-tech at ecs.soton.ac.uk'
>> Subject: [EP-tech] Re: Importing files from another system
>>
>> Nothing silly in the verbose output.
>>
>> I can wget the file over http (when trying the http method), but even
>> using the file from the local filesystem doesn't work :o\
>>
>>
>> -----Original Message-----
>> From: eprints-tech-bounces at ecs.soton.ac.uk
>> [mailto:eprints-tech-bounces at ecs.soton.ac.uk] On Behalf Of Yuri
>> Sent: 21 June 2012 13:48
>> To: eprints-tech at ecs.soton.ac.uk
>> Subject: [EP-tech] Re: Importing files from another system
>>
>> run the command with --verbose --verbose and try to see what happen. Can
>> you go to port 80 from the server to another server? Check your firewall
>> settings.
>>
>> Il 21/06/2012 14:22, John Salter ha scritto:
>>> Hi,
>>> I'm trying to import files from another system, into EPrints.
>>> I can either copy the files onto the server, or make them available
over
>>> http.
>>>
>>> In the XML, I've tried specifying (in an<eprints><eprint> node)
>>> <document>
>>> <file href="/path/to/my.pdf" />
>>> </document>
>>> - or -
>>> <document>
>>> <file>
>>> <url>http://url.of/my.pdf</url>
>>> </file>
>>> </document>
>>>
>>> And importing with:
>>>> bin/import ARCHIVENAME --enable-file-imports --enable-web-imports
>>>> --user 1 archive XML test.xml
>>> Both import the metadata from the XML OK, but neither option attaches
>>> the file to the eprint.
>>>
>>> What am I doing wrong - or will this even work?
>>>
>>> Cheers,
>>> John
>>> *** Options:
>>> http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
>>> *** Archive: http://www.eprints.org/tech.php/
>>> *** EPrints community wiki: http://wiki.eprints.org/
>> *** Options:
http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
>> *** Archive: http://www.eprints.org/tech.php/
>> *** EPrints community wiki: http://wiki.eprints.org/
>>
>> *** Options:
http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
>> *** Archive: http://www.eprints.org/tech.php/
>> *** EPrints community wiki: http://wiki.eprints.org/
>>
>> *** Options:
http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
>> *** Archive: http://www.eprints.org/tech.php/
>> *** EPrints community wiki: http://wiki.eprints.org/
>>
>> *** Options:
http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
>> *** Archive: http://www.eprints.org/tech.php/
>> *** EPrints community wiki: http://wiki.eprints.org/
>
>
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: http://www.eprints.org/tech.php/
> *** EPrints community wiki: http://wiki.eprints.org/
*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/