EPrints Technical Mailing List Archive

Message: #03390

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

[EP-tech] Re: Conversion of images, videos into preview/thumbnail versions

Hi Jacob,

The default charset is the charset used to create new tables and new fields. (e.g. when you add new fields to eprints).  It is best to change the default to utf8 (by using the ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_unicode_ci;). But this is not directly related to your problem.  

You need to make sure the existing tables' encoding  are in utf8.

To check your table encoding, in mysql command line, type:
SHOW FULL COLUMNS FROM tablename;

The Collation column tells you the current encoding used for the field. I presume you will find them in "Latin1".

If so, you will need to convert the table to utf8 encoding:
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;

After you converted tables, you will need to re-save the text as the utf8 texts are already corrupted.

Best
Jiadi





On 13/08/2014 15:30, Jacob Wenzel wrote:
Thanks for your reply Jiadi, but I think all my tables are already utf8, it is only default_character_set_name that is latin1.

All tables show utf8_general_ci in the column Collation when I run show table status from eprints_db_name;

That is why, I'm guessing I only need to run ALTER DATABASE eprints_db_name CHARACTER SET utf8 COLLATE utf8_general_ci;

It is not clear to me if you missed that piece of information or if I will need to convert all the tables anyway. Could you please clarify that?

Sorry for bugging you again. All the best!
/Jacob

On 13-08-2014 14:32, Jiadi Yao wrote:
Hi Jacob,

You only need to convert those tables that will store utf8 characters, but as a good practice, it is better to convert all tables to utf8.

You can try and find some tutorials that allow you to convert the whole database to utf8.

Regards,
Jiadi

On 13/08/2014 12:04, Jacob Wenzel wrote:
Thanks again Jiadi!

When I do:
SELECT default_character_set_name FROM information_schema.SCHEMATA WHERE schema_name = "eprints_db_name";

I get:
default_character_set_name: latin1

But when I do:
show table status from eprints_db_name;

I get:
Collation: utf8_general_ci

Knowing this, it enough for me to do the following?
ALTER DATABASE eprints_db_name CHARACTER SET utf8 COLLATE utf8_general_ci;

It seems that it would be quite time consuming to convert every single table in the EPrints DB one by one.

Actually I have never upgraded EPrints, I installed EPrints 3.3.12 through apt-get on Ubuntu Server 10.04 LTS and then transfered the DB and files to Ubuntu Server 12.04 LTS, when my VPS provider upgraded their systems. I followed this guide, I don't know if that is where the db confusion began.
http://wiki.eprints.org/w/Moving_a_repository

Thanks a lot for your help!

On 07-08-2014 11:42, Jiadi Yao wrote:
Hi Jacob,

This is to do with the encodings used by the database tables.

The latest version of eprints would create tables using UTF-8 encoding, but if you upgraded from an older version of eprints, you may still have the Latin encoding. UTF-8 characters will be messed up if you to save them into the a Latin encoding database table. 

To check the encoding:
SELECT default_character_set_name FROM information_schema.SCHEMATA WHERE schema_name = "eprints_db_name";

If it is not UTF-8, you will need to convert your database tables to UTF-8 encoding manually.  (http://stackoverflow.com/questions/6115612/how-to-convert-an-entire-mysql-database-characterset-and-collation-to-utf-8 )

Make sure you backup your database before doing so!


Best
Jiadi


On 07/08/2014 09:57, Jacob Wenzel wrote:
Perfect, it all works now.

I did notice s small bug related to UTF8 I guess.

If video file name contains special characters like "Æ, Ø, Å", no preview will be generated. But I guess that would be bad practice to do anyway.

Thanks a lot for helping me out!

Jacob.

On 05-08-2014 23:19, Jiadi Yao wrote:
Hi Jacob,

You first need to run

$ bin/epadmin redo_mime_type

then
$ bin/epadmin redo_thumbnails

Technical details : document.media_video_codec in the database needs to be valid before thumbnails are generated. This field is populated after a video file is upload when ffmpeg is installed, otherwise you need to redo_mime_type to back-fill those videos uploaded before installing ffmpeg.


Best
Jiadi

On 05/08/2014 18:56, Jacob Wenzel wrote:
Great!

I restarted apache and uploaded a video, and it now displays correctly with a thumbnail.

Now I only need to redo the ones I already uploaded.

I tried "bin/generate_abstracts" and "bin/epadmin redo_thumbnails [eprint id]" but neither had any effect.

What am I missing?

All the best
Jacob


On 05-08-2014 16:39, Jacob Wenzel wrote:
Thank you both!

I have done what Jiadi toild me to. Was that what you meant by configure ffmpeg seb?

If not, than what do I need to do now?

If yes, what is the correct step for me now? Restart apache and regenerate thumbnails?

I really appreciate your feedback!

/Jacob

On 05-08-2014 15:43, Jiadi Yao wrote:
After you installed ffmpeg, you'll need to changed the codec settings in Thumbnails.pm :

1. change  libfaac codec with libvo_aacenc
2. change  vorbis codec to libvorbis

as those codes are no longer supported by ffmpeg.

In addition, in lib/syscfg.d/invocations.pl
"-vpre default" parameter need to be removed


I'll update the github when I get a chance.

Best
Jiadi


On 05/08/2014 14:27, Sebastien Francois wrote:
Hi,

Yes - it's a core functionality: https://github.com/eprints/eprints/blob/3.3/perl_lib/EPrints/Plugin/Convert/Thumbnails.pm

Do you have ffmpeg set-up?

Seb.

On 05/08/14 13:50, Jacob Wenzel wrote:
Is it even possible to show image thumbnail previews of uploaded videos and images in EPrints 3.3.12?

I use coversheets and the youtube-dl packages from the Baza, which generates a thumbnail preview from PDFs or imports a screenshot from the video on Youtube and adds a link to the video.

But if I upload actual image files or videos, the file is depicted with and icon and no preview link is available.

I found some guides/plugins that promise to do this, but they all seem to be targeting different, older versions of EPrints, and I haven't been able to find recent ones.

Furthermore I would think that this was basic functionality that any user would want, so I don't why this functionality shouldn't be available "out the box".

This is the most recent document i have been able to find, but my "eprints_render" does not include the lines, that are mentioned in the guide, so I suspect is is targeting an older version.
http://wiki.eprints.org/w/How_to_Create_Cover_Images_in_EPrints_3

I also looked at PreviewPlus (only for older versions) and a few other links

Maybe my setup is wrong, but "epadmin test" shows "everything seems OK", so any help or pointers would be greatly appreciated. Where should I look, and how should I start.

Thank you all in advance!
--

Med venlig hilsen/Best regards

JACOB WENZEL
Projektleder

ENERGIAKADEMIET
Strandengen 1
8305 SAMSØ
+45 60 61 15 97
jw@energiakademiet.dk
www.energiakademiet.dk



*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/




*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/


-- 
Jiadi Yao
jy2e08@ecs.soton.ac.uk
EPrints Services
3081, Building 32
University of Southampton


*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/




*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/




*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/




*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/




*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/


-- 
Jiadi Yao
jy2e08@ecs.soton.ac.uk
EPrints Services
3081, Building 32
University of Southampton


*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/




*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/


-- 
Jiadi Yao
jy2e08@ecs.soton.ac.uk
EPrints Services
3081, Building 32
University of Southampton


*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/




*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/


-- 
Jiadi Yao
jy2e08@ecs.soton.ac.uk
EPrints Services
3081, Building 32
University of Southampton