Merge Duplicates causing Hoster to not respond

[stefbeme]

Yesterday I added new songs to my database. I rebuilt the database and I have now lost thousands of files. Upon rebuilding it is stating I have 30000+ duplicate files. When I click on the merge files button Hoster does nothing, it locks up and will not proceed. I have version 5.10 I have checked my settings according to your guidelines for Windows Performance Changes and they are correct. I use Windows 7 on the existing computer that is having this problem. Please advise.

[Roy Dennis]

The not responding may be due to the amount of duplicated files mentioned.

How did you add these new files to the database?
How many new files did you add ?

Was it from the Build database dialog using the Add folders button, Index Hard drive files dialog or the auto index new files feature.

Some things you can check, have a look in your Quarantine folder first to see if any of the files have been moved to there you can find that from the Tools Menu/Hoster Data Folder, Quarantine.
There are also a couple of text files in the data folder you can check should tell you the file path of where the files were "BuildSongErrors.txt"
Then look for "MergeSongDBLog.txt" file and open it. If you look at the date header that corresponds to the date you tried the merge and see if any songs are listed.

[stefbeme]

Originally yesterday, I added about 20 files to an existing folder, I rescanned the folder were the files were added. Then (of course during a show) I get a 911 from my husband that there were many songs missing. He went to tools>songs database>rebuild database>no resolution>I then rescanned each individual folder>no resolution>I then looked at my excluded folders>added a few folders that were questionable>no resolution at this point the amount of merge files has now grown, each time I did each of the above steps. When I attempted to merge, Hoster would then freeze (not responding). We rebooted, attempted to merge several times, each resulting in freezing Hoster.
I am assuming after reading your reply, that I am (and have always) been adding new files incorrectly so I may need the best direction on how to accomplish this.
I have searched the Quarantine folder, and no files have the song title, numbers only therefore am unable to determine if the files are included in this folder.
MergeSongDBLog.txt>the only file that exists is dated March 21, 2013 which does not correspond with my attempt on 8/9/13.
BuildSongErrors.txt - the missing files are located here and I have identified there are MANY files in this txt that are not appearing in my current listing in Hoster (this gives me hope)

[Roy Dennis]

First off I see nothing wrong with the way you added new songs providing they were new songs that had not been imported before, example on another computer.
If they were songs that already had a kmh header attached or a kma file imported using another computer this could cause conflict.

Are all your music files on an external drive ?
If they are first off I should exclude the C:\ drive, make sure there are no folders showing in the list of folders for the C: drive.

Have you at some time added songs using another computer ?
If so adding tracks this way can confuse Hosters Book Id or Disc ID numbering as one computer doesn't know what ids were used.

As there are no files listed in the "MergeSongDBLog.txt" file is probably because none have been merged.

You need to identify the folders the duplicated files are in. Open the "BuildSongErrors.txt" file and it should show something like: Book ID, Title, Artist, File Path, and and Type (Kma or KMH), note down the main file path excluding the title artist part, are most of them showing the same file path, in other words are the majority from the same folder ?
Hoping they are as this can make things easier.

Next note down a Book ID and type that book id into Hosters Book id field, does another song show for that same ID ? if so that is where the conflict is, add that song to playlist, right click on song in playlist and select edit from the dropdown.
This will show that song in the edit dialog, note the Disc Id, if it's a true disc Id copy it from the black disc id window at the bottom noting the track # then using Windows explorer paste that into the search box of the drive containing the song files and see if it finds that track, if it does note down the folder that the song is in.
If the disc Id is not a true disc Id like XYZ0xxx then you will have to just search on the title which can make it harder to identify if you have multiple choices but if you play the song from the playlist you should be able to recognise its brand thereby helping identify the track.
Now repeat the above with a couple more of the songs listed in the .txt file and see if the majority are in the same folder.
Hoping that they are exclude that folder for now from the rebuild list.

Now you say the files in your quarantine folder are all just numbers, I would assume in that case that they are kma types, if so open Hoster Tools Menu/Hoster Data Folder/Quarantine folder, do the files numbers correspond to any of the disc Id's in the .txt file?
If they do close Hoster but leave the quarantine folder open.

You need to copy these files back to their original folder, If they are kma types I would assume they are in one folder so this should be no problem.

Assuming things are as I stated and not masses of problem folders first rename the .txt file just in case we want to refer to it again for backup purposes, Hoster will ignore it if you rename it, start Hoster and rebuild the database making sure the problem folder or folders have been excluded, also make sure none of the "Song Types To Index" buttons are selected so that it will only index the previously indexed files. If there are still problem files it will re-create the .txt file.

See how you get on with that first then come back it may be you will get a few duplicated files but hopefully a lot less than previously.

If that goes alright I'll try and guide you through re-importing the problem folder.

[stefbeme]

Open the "BuildSongErrors.txt" file and it should show something like: Book ID, Title, Artist, File Path, and and Type (Kma or KMH), note down the main file path excluding the title artist part, are most of them showing the same file path, in other words are the majority from the same folder ?
Yes
Have you at some time added songs using another computer ? Yes, when I add songs to another computer I load them onto a flash drive, then add to a folder on that computer>rebuild database. We can look at this possibly later if this is causing confusion. I was not aware the two could speak to each other.
does another song show for that same ID ? No In fact much to my amazement, there are only about 120 songs showing in my Hoster>search
I have copied all the kma files from Quarantine to the C:/MTU/KMA files (existing file folder)
I am uncertain which is the exact problem folder so I chose to exclude two folders, rebuilding the main one with the KMA files.
Here are the result:
18340 Duplicate ID
8 files failed to index in the Database.
Click the ‘View Log File’ button to find them
on the hard drive.
You may need to re-import bad CD tracks,
or fix the bad
original mp3+g, wma+g, zip, audio, video files.
This contains file information:
Error file: C:\MTU\KMA Files\MMVE20-1 (2).kma
Error file: C:\MTU\KMA Files\MMVE20-1.kma
Error file: C:\MTU\KMA Files\PS1275-4 (2).kma
Error file: C:\MTU\KMA Files\PS1275-4.kma
Error file: C:\MTU\KMA Files\SC3458-9 (2).kma
Error file: C:\MTU\KMA Files\SC3458-9 (3).kma
Error file: C:\MTU\KMA Files\SC3458-9.kma
Error file: C:\MTU\KMA Files\SC9014-6.kma

Again, when I attempt to merge the duplicate file it has stalled Hoster (not responding)

And by the way I do want to mention thank you for your assistance on a weekend. It is very much appreciated!!!!!

[Beavis]

its not responding because its working on merging. it may take hours.

[Roy Dennis]

Originally Posted by stefbeme

In fact much to my amazement, there are only about 120 songs showing in my Hoster>search
I have copied all the kma files from Quarantine to the C:/MTU/KMA files (existing file folder)
I am uncertain which is the exact problem folder so I chose to exclude two folders, rebuilding the main one with the KMA files.
Here are the result:
18340 Duplicate ID
8 files failed to index in the Database.
Click the ‘View Log File’ button to find them
on the hard drive.
You may need to re-import bad CD tracks,
or fix the bad
original mp3+g, wma+g, zip, audio, video files.
This contains file information:
Error file: C:\MTU\KMA Files\MMVE20-1 (2).kma
Error file: C:\MTU\KMA Files\MMVE20-1.kma
Error file: C:\MTU\KMA Files\PS1275-4 (2).kma
Error file: C:\MTU\KMA Files\PS1275-4.kma
Error file: C:\MTU\KMA Files\SC3458-9 (2).kma
Error file: C:\MTU\KMA Files\SC3458-9 (3).kma
Error file: C:\MTU\KMA Files\SC3458-9.kma
Error file: C:\MTU\KMA Files\SC9014-6.kma

Again, when I attempt to merge the duplicate file it has stalled Hoster (not responding)

As beavis mentioned it can take a time to merge.
Is the 18340 the amount it indexed or the amount of duplicates ?

If you look at those 8 files they are all copies of one another except the last one, the figure in brackets at the end denotes it's a copy of the one without the brackets, the one SC3458-9 has 2 extra copies (2) & (3) if a lot of your files are like that you must be using lots of hard drive space. I should look in your kma folder and see if you have lots of these with the extra figure in brackets.

Anyway now that those kma files have merged do you think you are seeing all those 18340 files in your search dialog now, only you stated above you were amazed to only find 120 ?

When you import your files on your other computer do you import them as kma or kmh types ?

Is your other computer using version 5.10 ? if not what version is that using ?

Are the files in the folders that you excluded kma or kmh types ?

Do you still need to import the files in those two folders you excluded ?