User forums now online! Please see the latest post in the Announcements forum for more information.

Author Topic: What does "building file list" mean?  (Read 1041 times)

0 Members and 1 Guest are viewing this topic.

Mizkreant

  • Member
  • **
  • Posts: 11
    • View Profile
What does "building file list" mean?
« on: 2019-02-21 21:57:29 »
And why does Thumbsplus have to do it every time I switch folders? If all of the data and the thumbnails are in the database already, why is it doing this so often, and why does it take so long? I already allocated the maximum amount for the thumbnail cache (1000 apparently). I've tried storing the database on an SSD, but this didn't improve the speed or frequency of building the list. If this is unavoidable, perhaps future versions should have the option of storing more in RAM to avoid this.

Daan van Rooijen

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 832
    • View Profile
Re: What does "building file list" mean?
« Reply #1 on: 2019-02-21 22:21:53 »
It means that ThumbsPlus is reading the thumbnail records for that folder from its database, matching it to the image files on disk, and building the list so it can display it.

Things that may slow down the process include:

- When the folders and files are on a slow (network or external) drive
- When you have very large numbers of files or subfolders in a folder (e.g. 10,000)
- When you use Color Management in the thumbnail listing (Options | Preferences | tab:Thumbnail view | Enable Color Mgt and Options | Viewing | Color Management | Thumbnail View)
- When you display one or more User Fields in the thumbnail listing (press light bulb button above listing to select fields)

For many more suggestions see the ThumbsPlus Performance Hints on Cerious' website.

Anyway, rest assured that the program should (be able to) work much faster than you seem to be experiencing today.
I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

Mizkreant

  • Member
  • **
  • Posts: 11
    • View Profile
Re: What does "building file list" mean?
« Reply #2 on: 2019-02-21 22:39:54 »
The frequency with which it does this suggests that it doesn't cache very much in RAM. Also, the time it takes is inconsistent and I don't understand why. I have a folder which I work with regularly which has about 20,000 files. Most of the time building the file list only takes 5-10 seconds. Sometimes, however it has taken several minutes to display.

I've turned off Color Management for thumbnails. However, I still meet two of the criteria you listed for slowdowns. I have a large amount of files, and some of them are stored on an external drive. The database itself is stored on an internal HDD. As I said though, whether the database was placed on an SSD or HDD didn't seem to matter.

Is a SQLite3 database slower than MS Access? I've used Thumbplus 7 in the past and I seem to remember it being faster, though I could be mistaken.

Daan van Rooijen

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 832
    • View Profile
Re: What does "building file list" mean?
« Reply #3 on: 2019-02-22 02:50:50 »
I have a folder which I work with regularly which has about 20,000 files.

That's almost certainly the cause of the delays. With so many files, the folder will be read very slowly due to limitations in Windows. However, once it has been read, it will be temporarily stored in Windows' internal disk cache and then it will load faster during that session, until you, or a background program, reads some differrent folders that will then be cached at the expense of your first folder.

I don't think there's much that you can do about this. Defragmenting that drive may help for a while. Also, I believe there are third party drive caching tools that cache your HDD to RAM or to SSD, but I have no experience with them.

As for SQLite3, it used to be slow in earlier versions of TP, but it should offer very good performance in the current version 10 sp2.
I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

Mizkreant

  • Member
  • **
  • Posts: 11
    • View Profile
Re: What does "building file list" mean?
« Reply #4 on: 2019-02-22 07:26:41 »
been read, it will be temporarily stored in Windows' internal disk cache and then it will load faster during that session, until you, or a background program, reads some differrent folders that will then be cached at the expense of your first folder.
What setting in Windows controls this? Virtual Memory?

Daan van Rooijen

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 832
    • View Profile
Re: What does "building file list" mean?
« Reply #5 on: 2019-02-22 11:14:20 »
I don't think Windows offers you any controls over its disk caching. Maybe through the registry.. try Google.
I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

Mizkreant

  • Member
  • **
  • Posts: 11
    • View Profile
Re: What does "building file list" mean?
« Reply #6 on: 2019-02-22 14:59:40 »
I found an old piece of software from SysInternals called Cacheset which controls Windows' disk cache. Unfortunately it didn't help. The cache never seems to go above 700 MB no matter what its set to.

Street Mann

  • Full Member
  • ***
  • Posts: 51
    • View Profile
Re: What does "building file list" mean?
« Reply #7 on: 2019-03-04 10:42:09 »
....    I've used Thumbplus 7 in the past and I seem to remember it being faster, though I could be mistaken.

YES - T7 is blazing fast compared to t10 .. I just posted my tale of disappointment and then I saw that you were onto the same thing.

Street Mann

  • Full Member
  • ***
  • Posts: 51
    • View Profile
Re: What does "building file list" mean?
« Reply #8 on: 2019-03-04 10:54:12 »
I have a folder which I work with regularly which has about 20,000 files.
....  However, once it has been read, it will be temporarily stored in Windows' internal disk cache and then it will load faster during that session, until you, or a background program, reads some different folders that will then be cached at the expense of your first folder.


I'm not sure that this is entirely correct, but points to something that may be going on, not with windows but with T10 -  I suspect that it is NOT the entire file list of thumbs in the folder that is cached ( for display)  but only those that are going to be displayed in the current shown window.  I say this because if you wait for a spell,  thinking that OK ill give it some time to load all the thumbs into somewhere and then Ill start scrolling, you will be disappointed because as soon as you scroll "off" the current display, to say the next rows of images, T10 is going to lag as it seems to be rebuilding something and then displaying them... HOWEVER if you suffer thru the delayed display as you scroll a bit and wait and then scroll a bit more and wait, and scroll a bit more and wait ..... to the bottom of the folder of your images, THEN after that, the fast scroll back and forward and fast presentation of the thumbnails is as it was in T7.

But, then, if you click to another folder and then click back into the first folder you will have to suffer this delay and aggravation all over again.

Again -  T7 did not have this problem even on Win 10 machines.

« Last Edit: 2019-03-04 17:45:40 by Street Mann »

Mizkreant

  • Member
  • **
  • Posts: 11
    • View Profile
Re: What does "building file list" mean?
« Reply #9 on: 2019-03-06 04:47:27 »
YES - T7 is blazing fast compared to t10 .. I just posted my tale of disappointment and then I saw that you were onto the same thing.
It's good to know I wasn't imagining things. Thumbsplus 10 is very inefficient in terms of caching. I have 16 GB of RAM and TP only uses 700 MB with the max thumbnail cache size.

Daan van Rooijen

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 832
    • View Profile
Re: What does "building file list" mean?
« Reply #10 on: 2019-03-06 15:48:40 »
YES - T7 is blazing fast compared to t10 ..

Again, the bad performance that you two are seeing is not representative of v10 and is not inherent to v10. Its speed should be comparable to that of v7, and on my system (and countless others) it is.
I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

Street Mann

  • Full Member
  • ***
  • Posts: 51
    • View Profile
Re: What does "building file list" mean?
« Reply #11 on: 2019-03-06 16:09:37 »
What "system" you working on .. PC? I am on a HP Spectre 360 SDD 16gb ram images on the disk and also have images on USB external hard drive.

Update .. It seems that the default database will do the pictures folder on the HD faster that it will do the display on the same folder using the database that is on the External.. Let me move the DB to the computer and see what happens ....

« Last Edit: 2019-03-06 16:34:01 by Street Mann »

Daan van Rooijen

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 832
    • View Profile
Re: What does "building file list" mean?
« Reply #12 on: 2019-03-06 16:40:53 »
Let me move the DB to the computer and see what happens ....

;D

Put it on the SSD! It's going to be very slow on an external drive.
I'm volunteering as a moderator - I do not work for Cerious Software, Inc.

Street Mann

  • Full Member
  • ***
  • Posts: 51
    • View Profile
Re: What does "building file list" mean?
« Reply #13 on: 2019-03-06 18:21:35 »
Moved the DB to SDD .. No Joy ... BUT .. it is getting real crazy. When I select the default database and then go to a folder having a lot of images on it , said folder located on my SDD too, the build of the thumbnails and the scrolling is zippy fast ... If I set  the database to the one that I converted from my td4 and then look into the same folder that is on the SDD that I looked at before, the images load and scroll slow ...

The image folder is the same and I am just setting a different database to use as the default ... This folder has never been indexed by either database ...  Let me scan my External  drive using the default database and see what happens ... my problem i foresee is how to get my keywords etc over to the new database.


 I am running a TP db repair right now .. ill see if that changes anything.

Daan van Rooijen

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 832
    • View Profile
Re: What does "building file list" mean?
« Reply #14 on: 2019-03-06 23:48:50 »
my problem i foresee is how to get my keywords etc over to the new database.

If this only concerns JPG, TIF and/or PNG files, TP7 has an IPTC editor that lets you write your database keywords and annotations into the metadata headers of the actual image files (doesn't work for raw). When that's been done, you can create a new database in TP10, scan the image files, and whilst doing so have TP10 automatically copy the keywords and annotations into the respective database fields. There's a bit more to this procedure - holler if interested.

Anyway, it does sound like you have a problem in the converted TD4 database. If you want to look deeper into that (e.g. compare its structure to a new database), Alex Nolan's MDB Viewer Plus is a great (and free) tool for that purpose. Before doing all that, you could also try to simply re-scan your image folders (to rebuild the thumbnail records) and see if that helps.
« Last Edit: 2019-03-06 23:50:26 by Daan van Rooijen »
I'm volunteering as a moderator - I do not work for Cerious Software, Inc.