-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
1.0
-
None
Imported from http://bugs.musicbrainz.org/ticket/5790
Original ticket said:
I'm very upset with this software. I think it has just deleted my hole song library with no warning. I'm using ubuntu 10.04 and installed picard from the repository. What I've done:
1. Opened picard 2. Pressed "add folder" and selected my musics folder. They were in a NTFS partition. 3. Pressed "Lookup" it scanned for a long time and found tags for most songs, filling the right column. 4. Checked "Options->Rename files" and "Options->Move files" 5. Pressed "Save"
All my files have been moved to nowhere. The original folder was missing. Then I took a look at "Options->Options->Move files" and there was just a dot "." in the directory box.
BrianFreud replied:
Given that it's NTFS, and depending on the naming string used, it's also possible that the files were written to an alternate stream attached to the files' directory. (See http://manpages.ubuntu.com/manpages/jaunty/man8/ntfs-3g.8.html ; the NTFS-3G driver does support reading and writing to streams). I know I did accidentally somehow get Picard to write to an alternate stream once, back when I was still using Windows, though I don't recall exactly what I'd done to the naming string that managed to do it.
...
voiceinsideyou did a lot more investigating and found:
There is the sysinternals "streams" utility, for Windows. [0]
I believe in order to have got it to write to an ADS, you would have had to somehow make Picard write files using a file naming string that had a colon in it. I believe opening and writing to a file handle with "filename:adsname" is enough to make Picard do this.[1] I've tested this myself on an NTFS volume and it works through black magic.
So my guess is that when doing this on *Nix, if you're writing to an NTFS volume, perhaps it's very easy to break things if you have "Replace Windows-incompatible characters" disabled. Possibly Python and Picard just blindly tries to write files with : in them which could make things get hidden in ADSes.
Still not quite sure how you could get it to move an entire collection hidden in an ADS though; without a somewhat unlucky file naming string. Perhaps some time later I'll get VirtualBox/VMWare going and see if I can replicate something like this in Picard itself.
[0] http://technet.microsoft.com/en-us/sysinternals/bb897440 [1] http://robindavid.comli.com/?p=446
...
Indeed I was able to replicate this on Ubuntu using the ntfs-3g driver with streams_interface=windows (which is not the default). Witha file renaming string of ":" and move to ".", replace windows incompatible chars turned off.
Before:
voiceinsideyou@ubuntu:/media/ntfs-3g/4 Corners/The Foundations$ ls -al total 17332 drwxrwxrwx 1 root root 4096 2011-08-14 18:51 . drwxrwxrwx 1 root root 0 2010-02-13 13:24 .. -rwxrwxrwx 1 root root 4941952 2011-08-09 21:16 06-4 Corners-On the Downlow.mp3 -rwxrwxrwx 1 root root 4916277 2011-08-09 21:16 07-4 Corners-Push It Up.mp3 -rwxrwxrwx 1 root root 3582080 2011-08-09 21:16 08-4 Corners-Walk Right.mp3 -rwxrwxrwx 1 root root 4292736 2011-08-09 21:16 09-4 Corners-Strife.mp3Picard renames
D: 3077994176 18:51:57 Updating file <File #111 u'06-4 Corners-On the Downlow.mp3'> D: 3077994176 18:51:57 Saving file u'/media/ntfs-3g/4 Corners/The Foundations/06-4 Corners-On the Downlow.mp3' D: 3077994176 18:51:57 Updating file <File #112 u'07-4 Corners-Push It Up.mp3'> D: 3077994176 18:51:57 Updating file <File #113 u'08-4 Corners-Walk Right.mp3'> D: 3077994176 18:51:57 Updating file <File #114 u'09-4 Corners-Strife.mp3'> D: 3077994176 18:51:57 Moving file u'/media/ntfs-3g/4 Corners/The Foundations/06-4 Corners-On the Downlow.mp3' => u'/media/ntfs-3g/4 Corners/The Foundations/:/: (4).mp3' D: 3077994176 18:51:57 Updating file <File #111 u': (4).mp3'> D: 3077994176 18:51:57 Saving file u'/media/ntfs-3g/4 Corners/The Foundations/07-4 Corners-Push It Up.mp3' D: 3077994176 18:51:57 Moving file u'/media/ntfs-3g/4 Corners/The Foundations/07-4 Corners-Push It Up.mp3' => u'/media/ntfs-3g/4 Corners/The Foundations/:/: (5).mp3' D: 3077994176 18:51:57 Updating file <File #112 u': (5).mp3'> D: 3077994176 18:51:57 Saving file u'/media/ntfs-3g/4 Corners/The Foundations/08-4 Corners-Walk Right.mp3' D: 3077994176 18:51:57 Moving file u'/media/ntfs-3g/4 Corners/The Foundations/08-4 Corners-Walk Right.mp3' => u'/media/ntfs-3g/4 Corners/The Foundations/:/: (6).mp3' D: 3077994176 18:51:57 Updating file <File #113 u': (6).mp3'> D: 3077994176 18:51:57 Saving file u'/media/ntfs-3g/4 Corners/The Foundations/09-4 Corners-Strife.mp3' D: 3077994176 18:51:57 Moving file u'/media/ntfs-3g/4 Corners/The Foundations/09-4 Corners-Strife.mp3' => u'/media/ntfs-3g/4 Corners/The Foundations/:/: (7).mp3' D: 3077994176 18:51:57 Updating file <File #114 u': (7).mp3'>After
voiceinsideyou@ubuntu:/media/ntfs-3g/4 Corners/The Foundations$ ls -al total 4 drwxrwxrwx 1 root root 4096 2011-08-14 18:51 . drwxrwxrwx 1 root root 0 2010-02-13 13:24 ..The streams
G:\4 Corners\The Foundations\.: :/: (1).mp3:$DATA 6425332 :/: (2).mp3:$DATA 5574784 :/: (3).mp3:$DATA 1038472 :/: (4).mp3:$DATA 4941952 :/: (5).mp3:$DATA 4916277 :/: (6).mp3:$DATA 3582504 :/: (7).mp3:$DATA 4292736 :/:.mp3:$DATA 6012126So this is certainly possible.... likely... hmmm, not sure about that.
Picard can write files on NTFS partition to "alternate data streams"
-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
1.0
-
None
Imported from http://bugs.musicbrainz.org/ticket/5790
Original ticket said:
I'm very upset with this software. I think it has just deleted my hole song library with no warning. I'm using ubuntu 10.04 and installed picard from the repository. What I've done:
1. Opened picard 2. Pressed "add folder" and selected my musics folder. They were in a NTFS partition. 3. Pressed "Lookup" it scanned for a long time and found tags for most songs, filling the right column. 4. Checked "Options->Rename files" and "Options->Move files" 5. Pressed "Save"
All my files have been moved to nowhere. The original folder was missing. Then I took a look at "Options->Options->Move files" and there was just a dot "." in the directory box.
BrianFreud replied:
Given that it's NTFS, and depending on the naming string used, it's also possible that the files were written to an alternate stream attached to the files' directory. (See http://manpages.ubuntu.com/manpages/jaunty/man8/ntfs-3g.8.html ; the NTFS-3G driver does support reading and writing to streams). I know I did accidentally somehow get Picard to write to an alternate stream once, back when I was still using Windows, though I don't recall exactly what I'd done to the naming string that managed to do it.
...
voiceinsideyou did a lot more investigating and found:
There is the sysinternals "streams" utility, for Windows. [0]
I believe in order to have got it to write to an ADS, you would have had to somehow make Picard write files using a file naming string that had a colon in it. I believe opening and writing to a file handle with "filename:adsname" is enough to make Picard do this.[1] I've tested this myself on an NTFS volume and it works through black magic.
So my guess is that when doing this on *Nix, if you're writing to an NTFS volume, perhaps it's very easy to break things if you have "Replace Windows-incompatible characters" disabled. Possibly Python and Picard just blindly tries to write files with : in them which could make things get hidden in ADSes.
Still not quite sure how you could get it to move an entire collection hidden in an ADS though; without a somewhat unlucky file naming string. Perhaps some time later I'll get VirtualBox/VMWare going and see if I can replicate something like this in Picard itself.
[0] http://technet.microsoft.com/en-us/sysinternals/bb897440 [1] http://robindavid.comli.com/?p=446
...
Indeed I was able to replicate this on Ubuntu using the ntfs-3g driver with streams_interface=windows (which is not the default). Witha file renaming string of ":" and move to ".", replace windows incompatible chars turned off.
Before:
voiceinsideyou@ubuntu:/media/ntfs-3g/4 Corners/The Foundations$ ls -al total 17332 drwxrwxrwx 1 root root 4096 2011-08-14 18:51 . drwxrwxrwx 1 root root 0 2010-02-13 13:24 .. -rwxrwxrwx 1 root root 4941952 2011-08-09 21:16 06-4 Corners-On the Downlow.mp3 -rwxrwxrwx 1 root root 4916277 2011-08-09 21:16 07-4 Corners-Push It Up.mp3 -rwxrwxrwx 1 root root 3582080 2011-08-09 21:16 08-4 Corners-Walk Right.mp3 -rwxrwxrwx 1 root root 4292736 2011-08-09 21:16 09-4 Corners-Strife.mp3Picard renames
D: 3077994176 18:51:57 Updating file <File #111 u'06-4 Corners-On the Downlow.mp3'> D: 3077994176 18:51:57 Saving file u'/media/ntfs-3g/4 Corners/The Foundations/06-4 Corners-On the Downlow.mp3' D: 3077994176 18:51:57 Updating file <File #112 u'07-4 Corners-Push It Up.mp3'> D: 3077994176 18:51:57 Updating file <File #113 u'08-4 Corners-Walk Right.mp3'> D: 3077994176 18:51:57 Updating file <File #114 u'09-4 Corners-Strife.mp3'> D: 3077994176 18:51:57 Moving file u'/media/ntfs-3g/4 Corners/The Foundations/06-4 Corners-On the Downlow.mp3' => u'/media/ntfs-3g/4 Corners/The Foundations/:/: (4).mp3' D: 3077994176 18:51:57 Updating file <File #111 u': (4).mp3'> D: 3077994176 18:51:57 Saving file u'/media/ntfs-3g/4 Corners/The Foundations/07-4 Corners-Push It Up.mp3' D: 3077994176 18:51:57 Moving file u'/media/ntfs-3g/4 Corners/The Foundations/07-4 Corners-Push It Up.mp3' => u'/media/ntfs-3g/4 Corners/The Foundations/:/: (5).mp3' D: 3077994176 18:51:57 Updating file <File #112 u': (5).mp3'> D: 3077994176 18:51:57 Saving file u'/media/ntfs-3g/4 Corners/The Foundations/08-4 Corners-Walk Right.mp3' D: 3077994176 18:51:57 Moving file u'/media/ntfs-3g/4 Corners/The Foundations/08-4 Corners-Walk Right.mp3' => u'/media/ntfs-3g/4 Corners/The Foundations/:/: (6).mp3' D: 3077994176 18:51:57 Updating file <File #113 u': (6).mp3'> D: 3077994176 18:51:57 Saving file u'/media/ntfs-3g/4 Corners/The Foundations/09-4 Corners-Strife.mp3' D: 3077994176 18:51:57 Moving file u'/media/ntfs-3g/4 Corners/The Foundations/09-4 Corners-Strife.mp3' => u'/media/ntfs-3g/4 Corners/The Foundations/:/: (7).mp3' D: 3077994176 18:51:57 Updating file <File #114 u': (7).mp3'>After
voiceinsideyou@ubuntu:/media/ntfs-3g/4 Corners/The Foundations$ ls -al total 4 drwxrwxrwx 1 root root 4096 2011-08-14 18:51 . drwxrwxrwx 1 root root 0 2010-02-13 13:24 ..The streams
G:\4 Corners\The Foundations\.: :/: (1).mp3:$DATA 6425332 :/: (2).mp3:$DATA 5574784 :/: (3).mp3:$DATA 1038472 :/: (4).mp3:$DATA 4941952 :/: (5).mp3:$DATA 4916277 :/: (6).mp3:$DATA 3582504 :/: (7).mp3:$DATA 4292736 :/:.mp3:$DATA 6012126So this is certainly possible.... likely... hmmm, not sure about that.