Uploaded image for project: 'MusicBrainz Server'
  1. MusicBrainz Server
  2. MBS-10976

Unicode Private Use Area characters are stripped during edit process

    XMLWordPrintable

    Details

      Description

      When editing the following release, Unicode "private use" characters were silently stripped from the tracklist when navigating to the submission page: https://musicbrainz.org/edit/71525782

      I then tried to create an artist with the name "" at https://test.musicbrainz.org/artist/create, but it failed with the following ISE:

      Caught exception in MusicBrainz::Server::Controller::Artist->create "The edit could not be created. Exception (MusicBrainz::Server::Exceptions::DatabaseError): 23514 DBD::Pg::st execute failed: ERROR: new row for relation "artist" violates check constraint "only_non_empty"
      DETAIL: Failing row contains (1990049, 7b1f3365-f03d-4535-bbf7-c547f1943d90, , , null, null, null, null, null, null, null, null, null, , 0, 2020-07-21 05:23:19.510745+00, f, null, null). [for Statement "INSERT INTO artist (gender,begin_date_day,sort_name,begin_date_year,comment,type,begin_date_month,end_area,begin_area,end_date_month,gid,name,ended,area,end_date_year,end_date_day) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) RETURNING id" with ParamValues: 1=undef, 2=undef, 3='', 4=undef, 5='', 6=undef, 7=undef, 8=undef, 9=undef, 10=undef, 11='7b1f3365-f03d-4535-bbf7-c547f1943d90', 12='', 13='0', 14=undef, 15=undef, 16=undef] at lib/MusicBrainz/Server/Controller.pm line 82.
       at lib/Sql.pm line 335."
      

      As it turns out, there was a recent change made to `remove_invalid_characters` that included the following construct: `\p{Co}` - I believe this is causing the issue.

      My particular use case ("") would require unblocking of the following private use area: U+E000..U+F8FF.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              bitmap Michael Wiencek
              Reporter:
              jacobbrett Jacob Brett
              Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Packages

                  Version Package
                  2021-03-15