Handling exception characters

When you call a transcoder's conversion functions, therefore, you can request one of the following transcoding scopes (a TTranscoder::ETranscodingScope value) to indicate how the transcoder should handle exception characters:

• kNoRoundTrip converts any character without an exact representation in the target encoding system to a substitution character (TUnicodeSpecial::kReplacementCharacter for conversions to Unicode). This process produces text that cannot be directly converted back to the original character set, and might require manual editing to be usable. However, because this option produces the fastest transcoding, it might be more efficient for transcoding between character sets where the exception problems are minimal.
• kPartialRoundTrip converts all characters that have a representation in the target character set, but the representation does not have to be identical. For example, the Macintosh encoding system contains a single ligature character [xde ], which transcodes to Unicode as a two-character sequence, [f] and [i]. The transcoding operation stops when it reaches a character with no representation in the target set. When you transcode text using this option, the lengths of the original text string and the result text string may differ because the transcoded characters might not all have one-to-one mappings.
• kFullRoundTrip converts only characters that have a one-to-one mapping to the target character set. The transcoding operation stops when it reaches a character without an identical representation in the target set. For example, a Macintosh transcoder might define the [xde ] character as an exception character which is mapped to the [f] and [i] character sequence. When the transcoder reaches the [xde ] character, it stops transcoding; it returns the index of the [xde ] character within the source string to indicate where it stopped.

Transcoding data with the kNoRoundTrip option is the simplest mechanism because characters have a one-to-one mapping and you can easily calculate storage needs. If you want to transcode text using either the kPartialRoundTrip or kFullRoundTrip option, you must consider how you will allocate storage. Exception characters might expand into two characters, so calculating storage needs based on the length of the input text might not be adequate.

You can handle calculating storage by:

• Using TTranscoder::GetMaximumBytesPerCharacter to calculate the maximum number of bytes that each character in the input text might require for conversion, allowing for expanding characters. If the source character set contains any characters that expand to two characters during conversion, this function returns the value 2. This method is simple, but in most cases produces an output buffer that is longer than necessary.
• Using TTranscoder::GetBufferSize to determine the actual storage requirement. This function iterates through the input string to calculate the required storage, producing a more accurate result but requiring more computation time.

Using GetMaximumBytesPerCharacter

  TASCIITranscoder transcoder();
  
  TTextCount maxBytesPerChar;
  maxBytesPerChar = transcoder.GetMaximumBytesPerCharacter(
      TTranscoder::kPartialRoundTrip);
  
  TTextCount bufferLength = maxBytesPerChar * unicodeText.GetLength();
  unsigned char* returnChars = new unsigned char[bufferLength];
  
  TTextRange unicodeRange(0,unicodeText.Length());
  TTextCount numCharsConverted = transcoder.ExtractFromText(
      unicodeText, unicodeRange, returnChars, bufferLength,
      TTranscoder::kPartialRoundTrip);

  TASCIITranscoder transcoder;
  TTextRange unicodeRange(0,unicodeText.GetLength());
  
  TTextCount bufferLength = transcoder.GetBufferSize(unicodeText,
      unicodeRange, TTranscoder::kPartialRoundTrip);
  unsigned char*\ returnChars = new unsigned char[bufferLength];
  
  TTextCount numCharsConverted = transcoder.ExtractFromText(
      unicodeText, unicodeRange, returnChars, bufferLength,
      TTranscoder::kPartialRoundTrip);

  TASCIITranscoder transcoder;
  unsigned long stringLength = strlen(myString);
  
  TTextCount countCanConvert = transcoder.CanConvert(myString,
      stringLength, TTranscoder::kFullRoundTrip);
  
  if (countCanConvert < stringLength) {
      // flag non-convertible characters, print messages, etcetera...
  }