Wednesday, December 15, 2021

Quality check

Recently I ran a CueTools scan on an entire previous version of TLMC, as I was planning to do since I discovered I had this option. Not exactly the v.19, as it was already slightly dismantled (~25 albums replaced with better versions, renames applied), but something pretty close, say 99+%. The results were quite unexpected for me. If you're curious you can download full check logs (the checker might've also grabbed some extra files from bonuses) and a summary in both original text format and a table. Visual html+css table form uses some heuristics to transform summary text into color-coded quality representation, so it's not entirely precise. If in doubt refer to the detailed logs. You could also run the check yourself, although be prepared to let the program single-threadedly crunch numbers for a day or so, the bottleneck is the audio decode time, which runs at ~200x realtime on my CPU for the TTA codec. My logs slightly differ from the ones produced by the original Cuetools, because I patched it a little for my convenience, but the change only displays calculated offset in CTDB checks, which is not shown in the unmodified version.

The summary of the results is as follows: about 20% of all disc images fail verification. Here by "fail" I don't mean "disc was not found", I mean "was found to be different and unrepairable". Oh, and completely by coincidence the peak level of the supermajority of such rips is almost exactly 98%. And by another coincidence normalization to 98% is the default value in an admittedly turned-off-by-default setting in "Exact"AudioCopy. In addition to those there are another 15% of disc images, which also have peak level of 98% and return "disc not found" and it'd be fair to assume they would also fail to verify.

Lessons to be had here:
1. Not only it is your duty as a software developer to provide sensible defaults, doing only that and no more is clearly insufficient to produce reasonable outcomes. Out of 10 monkeys that see an unknown lever at least 3 will pull it and leave it there. Any option that does anything unpredictable to the dumbest possible user should be hidden behind a curtain that requires a certain intelligence level to bypass, which is enough to understand what the thing that was hidden actually does. In our particular case it could have been a domain-specific scripting language interpreter window for general purpose audio transforms, not a "please ruin my rips" on-off switch in plain sight. Of course there is another danger that by simply removing anything the unfortunate could use to shoot their feet off, instead of properly hiding it because it is easier to do so, you risk turning your software into iCrap.
2. We don't know how badly mangled these rips are.
Maybe they were improperly done and there are stutters or clicks or anything else that wasn't in the original CDs.
Maybe the only problem is the normalization, so it is displeasing on an intellectual level, but would not be audible as a defect.
3. In my defense those album versions were the only ones available, so the only alternative to including these rips was not to include them. I could not have done anything differently, had I known about it, but I wasn't aware of the problem and its extent. Now I am and so are you.

10 comments:

  1. I guess I need to rip and upload a few more CDs.

    ReplyDelete
    Replies
    1. https://nyaa.si/view/1468198
      https://nyaa.si/view/1468199
      https://nyaa.si/view/1468200
      https://nyaa.si/view/1468201
      https://nyaa.si/view/1468202
      https://nyaa.si/view/1468206
      https://nyaa.si/view/1468209
      https://nyaa.si/view/1468210
      https://nyaa.si/view/1468211
      https://nyaa.si/view/1468213
      https://nyaa.si/view/1468214
      https://nyaa.si/view/1468215
      https://nyaa.si/view/1468216

      Delete
  2. I should put up a detailed accurip result interpretation post some time later, but for now the most important thing I should note is that in the absense of other information "AR: rip not accurate" does not mean much, because it needs content + offset match to say "accurate" and correct content + wrong offset situation is mostly fine and trivial to fix.

    ReplyDelete
  3. Will this delay the v.20 release?

    ReplyDelete
    Replies
    1. Nope. The files will just stay, which requires no effort from me, until someone shares a better version.

      Delete
  4. Hey rwx, I don't know where to contact you elsewhere, but I have some albums rips that aren't already in TLMC. I also have some various scans of albums that are currently in TLMC, as I own them physically. Can you tell me where I can contact you further?

    ReplyDelete
    Replies
    1. You're already contacting me, I don't know how you can do it further. If you want a private contact method you can write to xmpp/jabber address rwx@headcounter.org, but just to donate rips you don't even need that - upload them somewhere and share a link in comments, preferably in the "Pre-release post" thread. Additional upside of that is that people can grab it themselves without waiting for the torrent, although in this case it's less important, as I think I'm wrapping up and plan to share the next version in several weeks [*]. I don't have a dedicated upload server, as most of the time I was operating in pull mode, if that's what you're interested in.

      [*] Nearly every single one of my previous estimates was off in a certain direction, take note.

      Delete
  5. Okay awesome. Thanks, and I'll get the rips and images to you as soon as possible.

    ReplyDelete
  6. [DiGiTAL WiNG]\2013.05.26 [DWCD-0008] BEST OF WiNG [例大祭10] is outright broken; at about 3:02 into the fifth track, it suddenly cuts into a different song and the final two tracks in the .cue don't play, as if someone had cut out eight minutes or so of the album right in the middle. The AccuRip indicates the same; the first four tracks are reported as accurately ripped and the rest of them fail entirely.

    ReplyDelete
    Replies
    1. I've found a non-broken version, will replace that file.

      Delete