Author Topic: Database size and Cloud Drives  (Read 6294 times)

0 Members and 1 Guest are viewing this topic.

bbelk

  • Member
  • **
  • Posts: 21
    • View Profile
Database size and Cloud Drives
« on: 2016-01-12 08:45:58 »
The size of the database has always been a issue when doing backups.  Mine is 1.3G.  I have moved my photos to a cloud drive, which seems to be working so far, but having to move 1.3G between my drive and the cloud every time I make a change is impractical as it eats up all my bandwidth for too long and sort of wipes out my dataplan when I am connecting through my phone.  Is there a way to separate the thumbnails from the database?
Is there a way to delete or diminish thumbnails for photos that have not been viewed in 60 days?
Thanks

Daan van Rooijen

  • Administrator
  • Sr. Member
  • *****
  • Posts: 933
    • View Profile
Database size and Cloud Drives
« Reply #1 on: 2016-01-12 19:47:54 »
> Is there a way to separate the thumbnails from the database?
> Is there a way to delete or diminish thumbnails for photos that have not been viewed in 60 days?


When you make a new database (Database | New) you can specify that you want thumbnails to be stored in separate files, outside of the database. I don't know how well this works though.

ThumbsPlus has no functionality to auto-delete inactive thumbnails, but you can delete them yourself by selecting them and pressing Shift-Del (or, right-click on a Drive in the tree panel and select Remove Thumbnails, or right-click on a folder and pick Remove from Tree). However, this does not physically remove the deleted thumbnails from the database - for that, you have to run Database | Advanced | Compact and Repair afterwards.
I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

bbelk

  • Member
  • **
  • Posts: 21
    • View Profile
Database size and Cloud Drives
« Reply #2 on: 2016-01-13 13:38:02 »
Can you create a new database from an old one?  I have 15 years worth of keywords I can't lose.

Is there a reason to think having separated thumbs may not work as well as having them in the database?

Daan van Rooijen

  • Administrator
  • Sr. Member
  • *****
  • Posts: 933
    • View Profile
Database size and Cloud Drives
« Reply #3 on: 2016-01-13 16:04:02 »
> Can you create a new database from an old one?  

I'm afraid not (except maybe with a lot of technical skill and patience.. it should be possible through Export/Import but that's fraught with issues, or by manipulating the database file directly).

> I have 15 years worth of keywords I can't lose.

I see. Do you have them only in the database or also in the files themselves (as IPTC keywords)? In that last case, rebuilding the database keywords would be easy.
 
> Is there a reason to think having separated thumbs may not work as well as having them in the database?

Not really, my hesitation stems from the fact that few people seem to have tried it, and I haven't tried it myself (and there's been little discussion of it here).

Also, it doesn't appeal to me personally because all those tiny separate files are going to consume a lot of disk space (each one will waste half a cluster on average), but that's just me being old-fashioned :)

I know that cloud storage is the future but the concept doesn't appeal to me. Just thumbnailing all those images means they have to be transfered back to the local computer on which you thumbnail them.. that must cost heaps of time and bandwidth. From an IT point of view, it would be more elegant if the database and image management software ran remotely in the cloud as well.
I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

bbelk

  • Member
  • **
  • Posts: 21
    • View Profile
Database size and Cloud Drives
« Reply #4 on: 2016-01-13 17:54:49 »
Tell me more about IPTC and keywords.  I have been looking for a way to do this. I need to save about 57,000 jpgs with keywords.

As for working off the cloud, it seems to work well as long as I keep the data base on the C: drive.  Opening the pictures (from my PC) does not generate network traffic unless I make a change to the file.

Daan van Rooijen

  • Administrator
  • Sr. Member
  • *****
  • Posts: 933
    • View Profile
Database size and Cloud Drives
« Reply #5 on: 2016-01-13 19:28:20 »
> Tell me more about IPTC and keywords.  I have been looking for a way to do this. I need to save about 57,000 jpgs with keywords.

IPTC is a standard that specifies a number of text fields that can be incorporated in the headers of some types of image files, mainly JPG, PNG and TIFF. One of these fields is Keywords (note: nowadays people usually talk about XMP data, which includes IPTC, EXIF and a few more types of metadata).

Through the Batch Edit Metadata command in TP10's Image menu, you can copy your database keywords into the IPTC Keywords field of the respective image files. The benefit of having them there is that the information will always travel with your images and that a database can be easily reconstructed (just re-scan the images and tell the program to read the keywords into its own database). A potential drawback is that anyone who has the image can see what information you have stored there.

It works like this: Select your image files, go to Image | Batch Edit Metadata, and set these values:

    [*] XMP field to modify: Description | Keywords
    [*] Operation to perform: Set (or Append if valuable keywords may already be present)
    [*] New Value: (press Get Value From button) Image Info | Keywords (press OK)
    [*] Now click Add Step, and press OK to apply the batch command.
    [/list]

    (Note that by writing to the image files, their time/date stamp will be updated to the current time).

    > As for working off the cloud, it seems to work well as long as I keep the data base on the C: drive.  Opening the pictures (from my PC) does not generate network traffic unless I make a change to the file.

    Well, if the file is stored remotely, it has to travel to your PC whenever you open it.. there's no way around that (except if the file is being cached because it had already been opened shortly before).
    I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

    bbelk

    • Member
    • **
    • Posts: 21
      • View Profile
    Database size and Cloud Drives
    « Reply #6 on: 2016-01-13 21:03:08 »
    This is great info on how to save the keywords.

    As for cloud stuff,  I am not a good source as I am new at this.  I can see my stuff from my mobile devices and from my phone it does have to load from online.  From my PC, there is a local copy on my hard drive and things work just as they did before the cloud, except that anything I change gets updated on the cloud so I have a constant rolling back up.  When I am just adding a few pictures, I don't notice the background traffic to the cloud, but when I exit from Thumbsplus after making a change, all of my bandwidth is sucked up by the copy being made to my 1.3G database.  It would really suck if I was camping and going through my phone's data plan.  So I keep my database off the cloud, which means I don't have a semi-live backup.

    That is why I am excited about getting the keywords into the jpegs.  It means that I will be able to recover from losing a database that I have been building since Windows 95.  I might lose my User Fields but I have made only limited use of them.  I lost my Galleries in the conversion from TP7 to TP8 several years ago.

    I will try to put keywords into jpegs in the morning.

    Daan van Rooijen

    • Administrator
    • Sr. Member
    • *****
    • Posts: 933
      • View Profile
    Database size and Cloud Drives
    « Reply #7 on: 2016-01-13 22:49:00 »
    Oh I see, the cloud storage acts as a back-up, but you also have your images locally. Sounds good!

    As for the user fields, you may be able to save those too. In the Metadata batch editor, you'd have to pick a suitable XMP field for each user field (eg. 'Comments'), and then you can populate that from the user field in question.

    If you have made new galleries, one thing that you could do it assign a unique keyword to all images in a gallery. For instance, if you have a 'Cars' gallery, you could assign "Cars-Gallery" to them. And then copy that to the IPTC keywords as well. Later on, if you should ever need to recreate your gallery, you can simply find all images that have that keyword and create a new gallery for them.
    I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

    bbelk

    • Member
    • **
    • Posts: 21
      • View Profile
    Database size and Cloud Drives
    « Reply #8 on: 2016-01-14 16:00:27 »
    This is fantastic info thank you so much.  Metadata has always been a mystery to me.

    How would I create a saved set of commands that would:

    Make Selected (adds folder names to the keywords)
    Saves the keywords into Metadata
    Remove Selected keyword (remove one keyword that happens to be "Scan Capture")



    This is the process I will be using as a final step when I am done processing new pictures and have renamed them and moved them to their permanent homes in the photo library.  The first and third steps above have been manual processes for the past 15 years.

    Daan van Rooijen

    • Administrator
    • Sr. Member
    • *****
    • Posts: 933
      • View Profile
    Database size and Cloud Drives
    « Reply #9 on: 2016-01-15 00:10:44 »
    > How would I create a saved set of commands that would:
    >
    > Make Selected (adds folder names to the keywords)
    > Saves the keywords into Metadata
    > Remove Selected keyword (remove one keyword that happens to be "Scan Capture")

    I'm afraid that can't be done through a batch command!

    For the metadata part, you can save the instructions to a set that you can later reuse through the Load button. See the top line in the Batch Edit Metadata dialog.

    Be sure to remove that Scan Capture keyword before copying the keywords into the files' metadata, though. You can either remove it for the currently selected files through Ctrl-K (enter -Scan Capture) or just delete the whole keyword from the database through Database | Edit Keywords.

    If you'd like to automate this further, I think you'll have to look into macro utilities for Windows such as AutoIt or Mouse Recorder.
    I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

    bbelk

    • Member
    • **
    • Posts: 21
      • View Profile
    Database size and Cloud Drives
    « Reply #10 on: 2016-01-15 08:29:54 »
    This process has worked pretty well through a few thousand pictures, but I am starting to get lots of this error:

    <5774> 011208-002.jpg: Unable to update XMP, error #000000ce:     Outrageous IFD count
    <5774> 011208-003.jpg: Unable to update XMP, error #000000ce:     Outrageous IFD count
    <5774> 011223-001.jpg: Unable to update XMP, error #000000ce:     Outrageous IFD count


    If I delete all metadata from the offending files, everything works fine, but with some lost information.

    Daan van Rooijen

    • Administrator
    • Sr. Member
    • *****
    • Posts: 933
      • View Profile
    Database size and Cloud Drives
    « Reply #11 on: 2016-01-15 15:55:15 »
    I don't know what might be causing that, but do these files have something in common, for instance were they all made with the same camera? Or, you notice something strange when you examine their EXIF info? (see File | Properties, then 3.Info tab).

    Maybe you can report this to Cerious (support@cerious.com - mention ThumbsPlus in your subject line) and submit one or a few of those images to them along with a brief description of what you did, what error you received and which TP and Windows version you use.

    IFD stands for Image File Directory and has to do with how EXIF information in a file is organized.
    I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

    bbelk

    • Member
    • **
    • Posts: 21
      • View Profile
    Database size and Cloud Drives
    « Reply #12 on: 2016-01-16 12:22:28 »
    This is an ongoing effort that is slowed down a lot by having the cloud back up going on.  Before I start again, I will turn off the wireless.

    Evidently there was some event on January 6, 2012 that impacted a large number of my photos.  For all that have this as a file date, the Metadata will not read or change.  When I look at properties in TP the number three tab (info) is missing.  TP will not modify the Metadata, but it will delete it and after that, everything is fine.

    Daan van Rooijen

    • Administrator
    • Sr. Member
    • *****
    • Posts: 933
      • View Profile
    Correction/Addendum
    « Reply #13 on: 2016-01-18 21:22:54 »
    If you want to try and convert your existing thumbnails into external files, this post offers a method (that I haven't tested):

    http://forums.cerious.com/forum/index.php?id=484
    I'm volunteering as a moderator - I do not work for Cerious Software, Inc.