EPrints Technical Mailing List Archive

Message: #02088

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

[EP-tech] Re: Memory usage in 3.2, Sword 1.3 and epdata packages

I also believe that the SWORD 2 implementation of the XML importer *does* replicate the SWORD 1.3 process: you can send it a complete file and it will do the CREATE and UPDATE processes in one go. 

On 12/07/13 10:51, Tim Brody wrote:

Correct. In 3.2 the HTTP post is all worked on in memory. In 3.3 XML
data are streamed and will be written to disk as it arrives.

/Tim.

On Fri, 2013-07-12 at 08:26 +0100, Ian Stuart wrote:

With no real knowledge, and certainly no investigation.... I would
suspect the problem is actually with how the base64 files are handled,
rather then being an EPrints memory leak per sae.

  From the SWORD importers I've written, the process seems to be to
1) read in the deposit
2) unpack the deposit (zip into disk space, XML into memory)
3) create the eprint object
4) attach the files
5) write everything out

So I would suspect that what's happening is that all your base64 files
are created (in memory) from the XML (which is also in memory)



--

Ian Stuart.
Developer: ORI, RJ-Broker, and OpenDepot.org
Bibliographics and Multimedia Service Delivery team,
EDINA,
The University of Edinburgh.

http://edina.ac.uk/

This email was sent via the University of Edinburgh.

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.