Ticket #295 (closed Defect: worksforme)

Opened 5 years ago

Last modified 5 years ago

movies/show entries duplicated in virtual folders after upgrade and subsequent full import

Reported by: sapphire_user Owned by:
Priority: normal Milestone:
Component: Other Version: 1.0b7
Keywords: Cc:

Description

Upgraded from b6.5 successfully. Upon Finder reboot, sapphire menu item was upgrade metadata which was completed successfully. All shows/movies correctly displayed in virtual folders with watched/unwatched/partially watched flags all intact.

Then added new files to ATV and did an import from the Sapphire main menu. In addition to adding new files to virtual folders it duplicated all the previous existing files as well. It appears that a full import subsequent to upgrading metadata does not treat upgraded metadata as already having been imported but rather as new items. New duplicated shows/movies reset watched flags to unwatched.

Please let me know if additional info is needed.

Thanks, Adam

Attachments

1-after upgrade.metaData.sapphireDataV2.zip Download (102.9 KB) - added by sapphire_user 5 years ago.
2-after import.metaData.sapphireDataV2.zip Download (119.4 KB) - added by sapphire_user 5 years ago.
Preview4_MetaData.zip Download (237.9 KB) - added by sapphire_user 5 years ago.
metaData.plist Download (148.7 KB) - added by sapphire_user 5 years ago.

Change History

comment:1 Changed 5 years ago by gbooker

  • Milestone set to Awaiting User Feedback

If a file is located at a different path than in the database, it is recorded as a separate file. You likely moved or renamed a directory without Sapphire detecting this was done.

Do you have symbolic links, like the ones that Nito floods in the Movies directory?

comment:2 Changed 5 years ago by sapphire_user

gbooker,

All media files are in the Movies directory however the Movies directory is located on a USB HDD attached to ATV via USB. I copied the contents of the Media mount point over to the external HDD and created a symlink from the ATV to the new Media location. Then I created a symlink in frontrow for Movies and pointed it to the external HDD as well. This way when iTunes looks at the ATV it sees the external HDD as my media drive. However, frontrow and OSBoot (as well as the symlinks) remain on the internal ATV drive. Sapphire does see both the symlink to Media (which is the top level directory above Movies within) as well as the direct link to Movies, however I set Sapphire not to import the collection pointing to the Media mount point, only the collection that points straight to the movies dir. I did this before the full import but not before the metadata upgrade since you don't see the Sapphire options menu item until after you upgrade the metadata. However, I assumed the metadata upgrade doesn't actually look at your media but only reconstitutes the old metadata.plist to V2. In any case, in previous versions of Sapphire when I did an import without telling Sapphire not to import from the Media mount point I did not see the duplication of items in the virtual folders.

Let me know if that answers your questions or if you need more info.

Thanks Adam

comment:3 Changed 5 years ago by sapphire_user

Also - did not move any media files on ATV/ext USB HDD from my last import under b6.5, through the upgrade to b7, the upgrade of the metadata to V2 or the full import.

Under b6.5 and earlier, when I did move files within the Movies dir Sapphire treated the file as new - it appeared to view the file in its original location as deleted and the file in its new location as a new file - no duplication appeared in the virtual folders when I moved or renamed files.

comment:4 Changed 5 years ago by gbooker

The virtual folders changed a bit in b7. If you have two copies of an episode or movie, whether they are copies of each other or completely different versions, both will show up in the virtual folder. In previous versions, you could have 1500 copies of one episode, and only one of them would show up in the virtual folder. Perhaps b6.5 had this issue, but it the effects were hidden.

Could you try the following: Go into settings, and into delete collections. Try deleting the collections one at a time and see if any one of them cause the duplicates to disappear. That could help narrow down how it is seeing two copies of a file. You will likely want to backup the metadata before doing this. Restoring the database is simply done by moving the file back and restarting the interface BTW, if it detects an old version of the database but not the current, it will present you with the upgrade option again.

comment:5 Changed 5 years ago by sapphire_user

gbooker,

You must be right about the 2 collections resulting in duplication in my case. I now recall that I stopped the initial full Import when I realized it was looking at the entire Media mount. I then set it not to import the Media collection and reran the import - however, I think that it had already finished looking in the Movie folder 2x (one in the Movies collection and one in the Media mount point).

Unfortunately, I've already downgraded to b6.5 (because of the AC3 passthrough issue) so I can't run the test. But, from your description how virtual folders behavior has changed, I'm sure you are right. I think you can chalk this up as 'invalid' I'll test it out on the next rev.

Many thanks for taking the time to fix the other issues and respond so quickly. Looking forward to the next rev.

Adam

comment:6 follow-up: ↓ 9 Changed 5 years ago by gbooker

  • Status changed from new to closed
  • Resolution set to invalid

OK, marking as invalid (I wish I had better choices in trac for resolved. Of Fixed, Invalid, Wontfix, Duplicate, and Worksforme, none really fit). If you should encounter this on the next version still, respond to this ticket and I'll re-open it.

comment:7 Changed 5 years ago by sapphire_user

gbooker,

I'll run the test you suggest above in the next rev. I'll comment back here with the result.

Adam

comment:8 Changed 5 years ago by sapphire_user

gbooker,

Thanks for the quick release of preview 2 - AC3 passthrough and quit UI now work for me.

The duplication after first full import remains, however. This time I made sure there was only one collection. Last time there was an alias to the mountpoint in the "Movies" folder (created by Nito's autolink) this time there was no such alias and so Sapphire correctly on sees one collection - the Movies folder. Once again metadata update completed without errors. Went back to Collections to confirm Sapphire still only seeing contents of my Movies folder. Then did full update. Dups back.

One thought. After metadata update, vid files still only show first audio track, even when there are 2. After full import, I can tell the original upgraded items from the new, imported items because the new items noq correctly show both audio tracks.

Attaching metadatav2 before and after import so you can see whats going on.

Again, thanks for the swift release of preview 2.

Adam

Changed 5 years ago by sapphire_user

Changed 5 years ago by sapphire_user

comment:9 in reply to: ↑ 6 Changed 5 years ago by sapphire_user

gbooker,

Replying since I'm not sure if merely commenting notified you the ticket was changed/amended. Duplication still present in Preview 2. Happily report that other defects I noted are properly squashed in latest version. Thanks for the quick release. See other posts and attachment for info on duplication error.

Adam

Replying to gbooker:

OK, marking as invalid (I wish I had better choices in trac for resolved. Of Fixed, Invalid, Wontfix, Duplicate, and Worksforme, none really fit). If you should encounter this on the next version still, respond to this ticket and I'll re-open it.

comment:10 follow-up: ↓ 11 Changed 5 years ago by gbooker

  • Status changed from closed to reopened
  • Resolution invalid deleted

Commenting is sufficient.

OK, I have to ask about the directory structure. I examined one movie: "Underworld Rise of the Lycans.m4v" (randomly selected).

Before import has the following path for this movie:
/mnt/Scratch/Users/frontrow/Movies/DVD/Underworld Rise of the Lycans.m4

After import has the following paths:
/mnt/Scratch/Users/frontrow/Movies/DVD/Underworld Rise of the Lycans.m4v
/mnt/Scratch/Volumes/kTV/Movies/DVD/Underworld Rise of the Lycans.m4v

So, is the DVD directory in frontrow's Movies directory a symlink? Is it something else. I'm assuming these are really the same file, so I'm wondering how these paths are related.

comment:11 in reply to: ↑ 10 Changed 5 years ago by sapphire_user

I see commenting was sufficient. Apologies for the duplication.

You are correct, movies is the symlink from frontrow and they really are the same file.

The attached USB drive is called KTV. It contains the physical location of "Movies" directory that the symlink in frontrow is pointing to. DVD and TV are sub directories of "Movies" Sapphire views this as two collections:

  1. /mnt/Scratch/Users/frontrow/Movies
  2. /mnt/Scratch/Volumes/kTV/Movies

Prior to doing the full import I went into options and told Sapphire not to import from: /mnt/Scratch/Volumes/kTV/Movies

The metadata from version b6.5 (upgraded by b7) would also have been importing frontrow/Movies, same location as where b7 did its import.

When I saw the duplications, i decided to let Sapphire b7 do its own import and skip the metadata upgrade. When I did a clean import (deleted metadata and metadatav2) the import worked correctly and virtual directories only include one entry for each media file.

Minor Historical Note: The reason for the symlink is that one of the earlier versions of Sapphire only looked in Movies (or at least needed to see a movies directory, if I recall correctly). I guess its not necessary anymore since Sapphire's been able to see media files in any mounts for quite some time now.

Adam

Replying to gbooker:

Commenting is sufficient.

OK, I have to ask about the directory structure. I examined one movie: "Underworld Rise of the Lycans.m4v" (randomly selected).

Before import has the following path for this movie:
/mnt/Scratch/Users/frontrow/Movies/DVD/Underworld Rise of the Lycans.m4

After import has the following paths:
/mnt/Scratch/Users/frontrow/Movies/DVD/Underworld Rise of the Lycans.m4v
/mnt/Scratch/Volumes/kTV/Movies/DVD/Underworld Rise of the Lycans.m4v

So, is the DVD directory in frontrow's Movies directory a symlink? Is it something else. I'm assuming these are really the same file, so I'm wondering how these paths are related.

comment:12 follow-up: ↓ 13 Changed 5 years ago by gbooker

Oh, you mean that /mnt/Scratch/Users/frontrow/Movies is a symlink to /mnt/Scratch/Volumes/kTV/Movies? That could be the culprit, as Sapphire may not detect this circumstance.

comment:13 in reply to: ↑ 12 Changed 5 years ago by sapphire_user

It is.

However, the problem only occurs on import after update. Didn't happen on clean import with no updated data or on subsequent imports since then.

Adam

comment:14 follow-up: ↓ 15 Changed 5 years ago by gbooker

(In [878]) Detect a Movies dir that is really a symlink upon upgrade of metadata. Refs #295

comment:15 in reply to: ↑ 14 Changed 5 years ago by sapphire_user

gbooker,

Again, many thanks for the rapid-fire updates and releases of the CorePreview?. Its a fantastic program that keeps getting better.

That being said, I reran the upgrade/full update process to see if changeset 878 fixed the duplication problem. Sadly it did not. I took my old metadata.plist from b6.5 and had the Preview 4 (7.3) run the upgrade and then did a full update. No problems with the upgrade - everything displays properly with one entry per media file. As before I made sure that Sapphire did not import from the direct volume path prior to running the full upgrade - but the full upgrade again resulted in duplicated entries. I've attached the latest versions of the upgraded V2 as well as the full import V2. As I mentioned before, this isn't a critical fix since the problem only occurs when importing AFTER upgrade. A clean import (achieved by deleting the plist) works fine -or, in this case, I just replaced the V2 file that I was using prior to this latest upgrade.

Adam

Replying to gbooker:

(In [878]) Detect a Movies dir that is really a symlink upon upgrade of metadata. Refs #295

Changed 5 years ago by sapphire_user

comment:16 follow-up: ↓ 17 Changed 5 years ago by gbooker

I have a theory. Could you include the metadata.plist so I could confirm it?

Changed 5 years ago by sapphire_user

comment:17 in reply to: ↑ 16 ; follow-up: ↓ 18 Changed 5 years ago by sapphire_user

Done.

Replying to gbooker:

I have a theory. Could you include the metadata.plist so I could confirm it?

comment:18 in reply to: ↑ 17 Changed 5 years ago by sapphire_user

Still duping on Preview 5. I'm wondering if I am now importing from a different collection than the original metadata.plist was using and that is what is causing the dups. Was that your theory as well? Unfortunately wasn't clever enough to figure out what I needed to do to confirm myself.

This is no longer a concern for me since I no longer use the original plist file for anything other than testing the new previous. I defer to you whether or not this should remain open or set to something like 'works for me'.

Many thanks, Adam

Replying to sapphire_user:

Done.

Replying to gbooker:

I have a theory. Could you include the metadata.plist so I could confirm it?

comment:19 Changed 5 years ago by gbooker

  • Status changed from reopened to closed
  • Resolution set to worksforme

Yeah, at this point I'm thinking it must have been a different collection than what was in the original metadata.plist. Oh well, at least you can use the thing.

comment:20 Changed 5 years ago by gbooker

  • Milestone Awaiting User Feedback deleted
Note: See TracTickets for help on using tickets.