I recently attempted to use the split artist credit feature:
Since the artist which was being split was credited as a different name on one release:
http://musicbrainz.org/release/8c46aace-36af-49f8-8bfd-95ca7ccab7c9 (track 2)
That artist credit was left unmodified. When I attempt to manually edit the artist credit for track 2 on that release now, any changes made to the artist credit are lost as soon as I leave the 'tracklist' tab on the release editor.