eprints_tech messages

From: Jean-Marie Le Bechec <Jean-Marie.LeBechec AT inp-toulouse.fr>
Date: Wed, 14 May 2008 16:25:08 +0200

*** http://www.eprints.org/tech.php/id/%3C482AF644.2010005%40inp-toulouse.fr%3E
*** EPrints community wiki - http://wiki.eprints.org/

hi,
	
I finally understood where the problem was. Records did not include 
publication date.

Is-it a bug ?

jean-marie

-------- Message original --------
Sujet: [EP-tech] Question about browse
Date: Fri, 11 Apr 2008 14:12:39 +0200
De: Jean-Marie Le Bechec <Jean-Marie.LeBechec AT inp-toulouse.fr>
Répondre à: EPrints.org Technical List <eprints-tech AT ecs.soton.ac.uk>
Organisation: INPT
Pour: EPrints.org Technical List <eprints-tech AT ecs.soton.ac.uk>

*** 
http://www.eprints.org/tech.php/id/%3C47FF55B7.1000203%40inp-toulouse.fr%3E
*** EPrints community wiki - http://wiki.eprints.org/

Hello.

I don't understand. Why browse (for some eprints) by author (for ex.) is
wrong ? With 0 item ?

ex :
http://oatao.univ-toulouse.fr/view/people/
http://oatao.univ-toulouse.fr/view/people/Asadulghani,_XX.html

My config : EPrints 3.0.3 (Sage)

Any idea ?

Thanks...

jean-marie

-- 

***********************************************
Jean Marie Le Bechec
Responsable ingenierie documentaire

Service Commun de la Documentation
Institut National Polytechnique de Toulouse
6 allee Emile Monso - bp 34038 -
31029 Toulouse cedex 4
Tel : 05 34 32 31 16
Mail : lebechec AT inp-toulouse.fr
***********************************************


-

From: Edward Craft <e.j.craft AT bham.ac.uk>
Date: Thu, 15 May 2008 12:26:02 +0100

*** ↵
http://www.eprints.org/tech.php/id/%3CF0BEFA7AEB684E41ABFF145F897C444FDA38B5%40EM4.adf.bham.ac.uk%3E
*** EPrints community wiki - http://wiki.eprints.org/

 
Hi,

We've just uploaded the first batch of 4000 metadata-only records from a
3rd party database into a 'pending repository', the idea being that user
can then upload files and then we can transfer them to a live repository
using export/import.

The records imported OK but now generate_views uses up all the server's
memory (5Gb + 2Gb swap : I inherited the server with this config) and
crashes.

The server is running ePrints 3.0.1b2 on RedHat ES3.

Checking the config, I'm still using DOM so I thought I'd change to
GDOME. First question - is this a possible cause of the memory problem ?

The server has gdome2-devel-0.8.1-1 and gdome2-0.8.1-1 installed.

The prerequisites XML-LibXML-Common-0.13 and XML-NamespaceSupport-1.09
installed OK but when I try to 'make' XML-GDOME-0.86 I get the error :

	rm -f blib/arch/auto/XML/GDOME/GDOME.so
	gcc  -shared GDOME.o dom.o  -o blib/arch/auto/XML/GDOME/GDOME.so
\
   		-L/usr/lib -L/usr/lib64 -lgdome -lglib -lxml2 -lz
-lpthread -lm      \

	/usr/bin/ld: skipping incompatible /usr/lib/libgdome.so when
searching for -lgdome
	/usr/bin/ld: skipping incompatible /usr/lib/libgdome.a when
searching for -lgdome
	/usr/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../libgdome.so 
		when searching for -lgdome
	/usr/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../libgdome.a 
		when searching for -lgdome
	/usr/bin/ld: skipping incompatible /usr/lib/libgdome.so when
searching for -lgdome
	/usr/bin/ld: skipping incompatible /usr/lib/libgdome.a when
searching for -lgdome
	/usr/bin/ld: cannot find -lgdome
	collect2: ld returned 1 exit status
	make: *** [blib/arch/auto/XML/GDOME/GDOME.so] Error 1

Question 2 : Is the problem the version of gdome2 (someone said they
recommended 0.7.x ?) and, if so, does anything else in ePrints use
gdome2 that might be afected by a reinstall ?


Any suggestions welcomed !!!

Edward.

Edward Craft
Digital Library Systems Specialist
Information Services
University of Birmingham
e.j.craft AT bham.ac.uk
 

From: Tim Brody <tdb01r AT ecs.soton.ac.uk>
Date: Thu, 15 May 2008 12:58:58 +0100

*** http://www.eprints.org/tech.php/id/%3C482C2582.80801%40ecs.soton.ac.uk%3E
*** EPrints community wiki - http://wiki.eprints.org/

Jean-Marie Le Bechec wrote:
> *** 
> ↵
http://www.eprints.org/tech.php/id/%3C482AF644.2010005%40inp-toulouse.fr%3E 
>
> *** EPrints community wiki - http://wiki.eprints.org/
>
> hi,
>     
> I finally understood where the problem was. Records did not include 
> publication date.
>
> Is-it a bug ?
Hmm, I think the records should still have appeared as "UNDEFINED".

The browse view generation has completely changed in 3.1, so I don't 
want to spend a lot of time on this unless you find out where the 
problem lies.

All the best,
Tim.

From: Tim Brody <tdb01r AT ecs.soton.ac.uk>
Date: Thu, 15 May 2008 13:04:51 +0100

*** http://www.eprints.org/tech.php/id/%3C482C26E3.70708%40ecs.soton.ac.uk%3E
*** EPrints community wiki - http://wiki.eprints.org/

Edward Craft wrote:
> *** ↵
http://www.eprints.org/tech.php/id/%3CF0BEFA7AEB684E41ABFF145F897C444FDA38B5%40EM4.adf.bham.ac.uk%3E
> *** EPrints community wiki - http://wiki.eprints.org/
>
>  
> Hi,
>
> We've just uploaded the first batch of 4000 metadata-only records from a
> 3rd party database into a 'pending repository', the idea being that user
> can then upload files and then we can transfer them to a live repository
> using export/import.
>
> The records imported OK but now generate_views uses up all the server's
> memory (5Gb + 2Gb swap : I inherited the server with this config) and
> crashes.
>
> The server is running ePrints 3.0.1b2 on RedHat ES3.
>   
I think there were a few XML::DOM related memory leaks that were fixed 
in 3.0.3. Quite a few things have been fixed since 3.0.1b2.
(Any reason for not upgrading to 3.0.5?)

GDOME will probably avoid the memory leaks and will be a lot faster anyway.

Have you tried following instructions on the wiki for gdome?
http://wiki.eprints.org/w/Installing_GDOME_on_Fedora_Core_3

All the best,
Tim.

From: Ben Wheeler <b.wheeler AT ulcc.ac.uk>
Date: Thu, 15 May 2008 13:17:35 +0100

*** ↵
http://www.eprints.org/tech.php/id/%3C20080515121735.GA16191%40brucia.ulcc.ac.uk%3E
*** EPrints community wiki - http://wiki.eprints.org/

On Thu, May 15, 2008 at 12:26:02PM +0100, Edward Craft wrote:
> The records imported OK but now generate_views uses up all the server's
> memory (5Gb + 2Gb swap : I inherited the server with this config) and
> crashes.
> 
> The server is running ePrints 3.0.1b2 on RedHat ES3.
> 
> Checking the config, I'm still using DOM so I thought I'd change to
> GDOME. First question - is this a possible cause of the memory problem ?

Yes, we found very severe memory leaks with EPrints using XML::DOM.

I would suggest using XML::LibXML rather than GDOME if possible,
as the latter's development has been stagnant for years.

Ben

From: Tim Brody <tdb01r AT ecs.soton.ac.uk>
Date: Thu, 15 May 2008 13:35:16 +0100

*** http://www.eprints.org/tech.php/id/%3C1210854916.8268.11.camel%40meteor%3E
*** EPrints community wiki - http://wiki.eprints.org/

On Thu, 2008-05-15 at 13:17 +0100, Ben Wheeler wrote:
> *** ↵
http://www.eprints.org/tech.php/id/%3C20080515121735.GA16191%40brucia.ulcc.ac.uk%3E
> *** EPrints community wiki - http://wiki.eprints.org/
> 
> On Thu, May 15, 2008 at 12:26:02PM +0100, Edward Craft wrote:
> > The records imported OK but now generate_views uses up all the ↵
server's
> > memory (5Gb + 2Gb swap : I inherited the server with this config) and
> > crashes.
> > 
> > The server is running ePrints 3.0.1b2 on RedHat ES3.
> > 
> > Checking the config, I'm still using DOM so I thought I'd change to
> > GDOME. First question - is this a possible cause of the memory ↵
problem ?
> 
> Yes, we found very severe memory leaks with EPrints using XML::DOM.
> 
> I would suggest using XML::LibXML rather than GDOME if possible,
> as the latter's development has been stagnant for years.

Hi Ben,

XML::LibXML requires EPrints 3.0.5 and will require the current CPAN
version, rather than the one that will be on ES3 that Edward is running.

Cheers,
Tim.

From: Ben Wheeler <b.wheeler AT ulcc.ac.uk>
Date: Thu, 15 May 2008 14:19:04 +0100

*** ↵
http://www.eprints.org/tech.php/id/%3C20080515131904.GC16399%40brucia.ulcc.ac.uk%3E
*** EPrints community wiki - http://wiki.eprints.org/

On Thu, May 15, 2008 at 01:35:16PM +0100, Tim Brody wrote:
> XML::LibXML requires EPrints 3.0.5 

It does? 
We're using it with 3.0.3 and 3.0.2b3, is there a known problem with that?

Ben

From: Edward Craft <e.j.craft AT bham.ac.uk>
Date: Thu, 15 May 2008 14:40:13 +0100

*** ↵
http://www.eprints.org/tech.php/id/%3CF0BEFA7AEB684E41ABFF145F897C444FDA38B9%40EM4.adf.bham.ac.uk%3E
*** EPrints community wiki - http://wiki.eprints.org/

I was just being lazy, really, and waiting for 3.1 before upgrading.
Instead I'll go to 3.0.5 and see if the generate_views runs OK after
that.

I'll still presevere with GDOME, though.

Thanks ! 

Edward 

-----Original Message-----
From: owner-eprints-tech AT ecs.soton.ac.uk
[mailto:owner-eprints-tech AT ecs.soton.ac.uk] On Behalf Of Tim Brody
Sent: 15 May 2008 13:05
To: EPrints.org Technical List
Subject: Re: [EP-tech] Gdome install question.

***
http://www.eprints.org/tech.php/id/%3C482C26E3.70708%40ecs.soton.ac.uk%3
E
*** EPrints community wiki - http://wiki.eprints.org/

Edward Craft wrote:
> *** 
> http://www.eprints.org/tech.php/id/%3CF0BEFA7AEB684E41ABFF145F897C444F
> DA38B5%40EM4.adf.bham.ac.uk%3E
> *** EPrints community wiki - http://wiki.eprints.org/
>
>  
> Hi,
>
> We've just uploaded the first batch of 4000 metadata-only records from

> a 3rd party database into a 'pending repository', the idea being that 
> user can then upload files and then we can transfer them to a live 
> repository using export/import.
>
> The records imported OK but now generate_views uses up all the 
> server's memory (5Gb + 2Gb swap : I inherited the server with this 
> config) and crashes.
>
> The server is running ePrints 3.0.1b2 on RedHat ES3.
>   
I think there were a few XML::DOM related memory leaks that were fixed
in 3.0.3. Quite a few things have been fixed since 3.0.1b2.
(Any reason for not upgrading to 3.0.5?)

GDOME will probably avoid the memory leaks and will be a lot faster
anyway.

Have you tried following instructions on the wiki for gdome?
http://wiki.eprints.org/w/Installing_GDOME_on_Fedora_Core_3

All the best,
Tim.

From: Tim Brody <tdb01r AT ecs.soton.ac.uk>
Date: Thu, 15 May 2008 14:59:16 +0100

*** http://www.eprints.org/tech.php/id/%3C1210859956.8268.13.camel%40meteor%3E
*** EPrints community wiki - http://wiki.eprints.org/

On Thu, 2008-05-15 at 14:19 +0100, Ben Wheeler wrote:
> *** ↵
http://www.eprints.org/tech.php/id/%3C20080515131904.GC16399%40brucia.ulcc.ac.uk%3E
> *** EPrints community wiki - http://wiki.eprints.org/
> 
> On Thu, May 15, 2008 at 01:35:16PM +0100, Tim Brody wrote:
> > XML::LibXML requires EPrints 3.0.5 
> 
> It does? 
> We're using it with 3.0.3 and 3.0.2b3, is there a known problem with that?

You might have a few presentational issues. Nothing critical AFAIK.

Cheers,
Tim.

From: Edward Craft <e.j.craft AT bham.ac.uk>
Date: Fri, 16 May 2008 09:22:40 +0100

*** ↵
http://www.eprints.org/tech.php/id/%3CF0BEFA7AEB684E41ABFF145F897C444FDA38BB%40EM4.adf.bham.ac.uk%3E
*** EPrints community wiki - http://wiki.eprints.org/

Hi,

I upgraded to 3.0.5 but had the same problem - memory usage rising to
100% within 20 mins and the server locking up.

After failing to get anywhere with GDOME I installed LibXML without a
problem and generate_views now peaks at 1.8% memory usage. Result ! I've
tried all the usual admin tasks (importing, generating
abstracts/static/views, reloading) and all seems OK.

Many thanks !!!

Edward.


-----Original Message-----
From: owner-eprints-tech AT ecs.soton.ac.uk
[mailto:owner-eprints-tech AT ecs.soton.ac.uk] On Behalf Of Tim Brody
Sent: 15 May 2008 14:59
To: EPrints.org Technical List
Subject: Re: [EP-tech] Gdome install question.

***
http://www.eprints.org/tech.php/id/%3C1210859956.8268.13.camel%40meteor%
3E
*** EPrints community wiki - http://wiki.eprints.org/

On Thu, 2008-05-15 at 14:19 +0100, Ben Wheeler wrote:
> *** 
> http://www.eprints.org/tech.php/id/%3C20080515131904.GC16399%40brucia.
> ulcc.ac.uk%3E
> *** EPrints community wiki - http://wiki.eprints.org/
> 
> On Thu, May 15, 2008 at 01:35:16PM +0100, Tim Brody wrote:
> > XML::LibXML requires EPrints 3.0.5
> 
> It does? 
> We're using it with 3.0.3 and 3.0.2b3, is there a known problem with
that?

You might have a few presentational issues. Nothing critical AFAIK.

Cheers,
Tim.