EPrints Technical Mailing List Archive

Message: #04647

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

[EP-tech] Re: Temporarily Disabling Deposits/Edits during migration/upgrade.

Hi Brian,

This is what I do (in brief):

1) Reduce the TTL on the repository DNS name to something like 5 minutes

2) "Freeze" the old repository - prevent users logging in and changing records

3) Migrate the repository to the new server - I use rsync to transfer the code and documents tree and mysqldump to transfer a snapshot of the database

After rsyncing, don't forget to remove any code changes that you added in step 2 (ie. you don't want the new repository to be frozen).

4) On the new server, increase the accessid counter eg.

mysql> update counters set counter=10000000 where countername="accessid";

The idea is to leave a "gap" in the access table for any downloads that occur during the migration, so round up the counter to a suitably high value.

5) Test that everything is working on the new server (add an entry to your /etc/hosts so you can access it using the repository URL)

Remember to remove the entry from /etc/hosts afterwards!

6) Change DNS to point to the new server

Within a few minutes traffic will start to hit the new server - your users will now be using the new server. Any new downloads will start from accessid 10000000 (or whatever you set the accessid counter to in step 4).

Some services (crawlers etc.) seem to cache DNS entries for up to 24+ hours so expect traffic to continue hitting your old server for a while.

7) Update access table on new server

During the migration/DNS caching period both the old and new servers will have been accumulating downloads in their respective access tables.

But - as we increased the accessid counter on the new server we can merge the old access table into the new without accessids clashing.

Dump the access table from the old server and copy it to the new server.

Create a new database on the new server and import the access table into it:

mysqladmin create tmp_access_update;

mysql tmp_access_update < access_table_from_old.sql

Then merge the tables:

mysql> use new;

mysql> select max(accessid) from access where accessid < 10000000; # or whatever you set the accessid counter to in step 4

6782323 # the highest accessid we imported from the old server

mysql> use tmp_access_update; # data from old server

mysql> select max(accessid) from access;

6791234 # compare to above - old server has accumulated data since we took the snapshot

mysql> insert into new.access values (select * from tmp_access_update.access where accessid>6782323); # inserts 6782324-6791234

8) Restore the TTL on the DNS - at some point later

Hope that helps,


Timothy Miles-Board

Web & Repositories Development Specialist, University of London Computer Centre

020 7863 1342  |  07742 970 351  | timothy.miles-board@london.ac.uk | @drtjmb

The University of London is an exempt charity in England and Wales

From: eprints-tech-bounces@ecs.soton.ac.uk <eprints-tech-bounces@ecs.soton.ac.uk> on behalf of Brian D. Gregg <bdgregg@pitt.edu>
Sent: 03 September 2015 7:55 PM
To: eprints-tech@ecs.soton.ac.uk
Subject: [EP-tech] Re: Temporarily Disabling Deposits/Edits during migration/upgrade.

Peter, John,


Both are very good answers, and yes the access table will continue to grow.  We’ll have to determine if the access table growth will be a concern as obviously it will affect our IRstats install.


Thanks again.




From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of John Salter
Sent: Thursday, September 03, 2015 2:23 PM
To: eprints-tech@ecs.soton.ac.uk
Subject: [EP-tech] Re: Temporarily Disabling Deposits/Edits during migration/upgrade.


Hi Brian,

In the past I've been advised to alter the user_login method to return 0 (so no one can log in), and then to remove the login tickets (so current sessions aren't valid).


This approach obviously depends on what login methods you've got configured for your repository.


Also worth noting that the access table will continue to grow even if people can't do stuff...





From: eprints-tech-bounces@ecs.soton.ac.uk <eprints-tech-bounces@ecs.soton.ac.uk> on behalf of Brian D. Gregg <bdgregg@pitt.edu>
Sent: 03 September 2015 17:27
To: 'eprints-tech@ecs.soton.ac.uk'
Subject: [EP-tech] Temporarily Disabling Deposits/Edits during migration/upgrade.




Sorry if this may be repeat question to the list but I’m not finding what I believe I’m looking for in the archives.


We will be migrating our EPrints system from one system to another and performing and upgrade in the process.  We’d like to keep the initial system up while we do so.  However, we want to disable any additional deposits/changes to the system while the migration is happening but want to keep the system up for viewing, searching, etc.   Due to the amount of content and the upgrade we’ll also be doing this will take at a minimum of 4 hours.  My initial thought is to comment out the permissions in the user_roles.pl file that apply and restart the web server to “lockout” any user updates.  If this approach is used what is minimum permissions needed to allow basic use/functionality by the user?  Is it just “general”?


Is this the correct approach or is there something simpler that we can do to achieve the same functionality?

Or would it be highly recommended to block the whole site while the migration is happening?


Thoughts, experiences?





Brian D. Gregg

Solutions Architect | Manager Systems Development

University of Pittsburgh | University Library System

Address: 7500 Thomas Blvd.  Room 129 Pittsburgh, PA 15208

Tel: (412) 648-3264 | Email: bdgregg@pitt.edu | Fax: (412) 648-3585