AnyDAC
ContentsIndexHome
PreviousUpNext
Unicode Support

Describes how AnyDAC works with Unicode data and metadata. AnyDAC fully support Unicode data and metadata. Although an application may need to take the additional steps to use Unicode properly.

Group
Links
General

Most enterprise-class database applications must be able to work with character data encoded in different character sets or provided as Unicode. AnyDAC provides seamless support for:

  • different single byte client character sets and code pages, including standard ANSI;
  • as well multi-byte strings such as Unicode, including UTF8, UTF16 and UCS2 encodings.

Note, that the complete Unicode support may be achieved only with the Unicode Delphi version usage - Delphi 2009 and higher. And application is free to use the Delphi native ANSI or Unicode strings. AnyDAC will perform transparent character set conversion between Delphi application and DBMS character set. 

In AnyDAC, UTF-16 encoded Unicode characters may be used for the following elements:

  • SQL command text;
  • SQL script text;
  • metadata values;
  • result set field values;
  • parameter values;
  • text data files;
  • trace output.

 

AnyDAC's handling of Unicode depends on:

  • the version of Delphi being used
  • client character set - UTF8/UTF16 or ACP compatible
  • DBMS driver.

 

Configuring connection definitions for Unicode

Note: It is recommended to set the DBMS client character set to Unicode, for Delphi 2009 and higher, to avoid conversion loss. 

In general, to configure connection definitions for Unicode, it is required to set the DBMS client character set to Unicode. How this is done depends on the version of Delphi and the used DBMS driver:

Driver 
Parameter 
Advantage Database Server 
Not supported. 
IBM DB2 Server 
Automatically set to UTF16 in Delphi 2009 and higher. Not supported in Delphi 2007 or less. 
Interbase or Firebird Server 
CharacterSet=utf8 
Microsoft SQL Server 
Automatically set to UTF16 in Delphi 2009 and higher. Not supported in Delphi 2007 or less. 
MS Access Database 
Automatically set to UTF16 in Delphi 2009 and higher. Not supported in Delphi 2007 or less. 
MySQL Server 
CharacterSet=utf8 
Oracle Server 
  • CharacterSet=utf8;
  • or NLS_LANG=_.UTF8, if CharacterSet is not specified.
 
Postgre SQL 
CharacterSet=utf8 
SQLite database 
Automatically set to UTF16 in Delphi 2009 and higher. Not supported in Delphi 2007 or less. 
Sybase SQL Anywhere 
Automatically set to UTF16 in Delphi 2009 and higher. Not supported in Delphi 2007 or less. 
dbExpress bridge driver 
Depends on dbExpress driver. 
ODBC bridge driver 
Automatically set to UTF16 in Delphi 2009 and higher. Not supported in Delphi 2007 or less. 

 

SQL command text

Starting with Delphi 2009, TADQuery.SQL, TADCommand.CommandText, IADPhysCommand.CommandText, and other property values are Unicode encoded. The SQL command text preprocessor supports only UCS2 encoding, which means that surrogate pairs are not supported. 

Before sending SQL command text to the DBMS, AnyDAC converts it to:

  • Active Code Page, if the client character set is SBCS (non Unicode). If the client character set is not compatible with ACP, conversion loss is possible.
  • UTF8, if client character set is UTF8.
  • UTF16, if client character set is UTF16.

To specify Unicode encoded SQL command text, just do the following: 

 

ADQuery1.SQL.Text := 'select ''Hello world !'' where Lang = ''RU'' and Val = ''Привет мир !''';

 

SQL script text

Starting with Delphi 2009, TADScript.SQLScripts collections can contain Unicode encoded SQL script texts. On prior Delphi versions TADScript can load Unicode encoded SQL script files, but they will be converted into ACP encoding before processing. Other than that, SQL script text processing is similar to SQL command text processing. 

To control SQL script file and log file encoding, use TADScript.ScriptOptions.FileEncoding property: 

 

ADScript1.ScriptOptions.FileEncoding := enUTF16;
ADScript1.SQLScriptFileName := 'u:\builddb.sql';
ADScript1.ExecuteAll;

 

Metadata values

Note: At the moment, AnyDAC does not support Unicode encoded column values in metadata result sets, returned by TADMetaInfoCommand, TADMetaInfoQuery and IADPhysMetaInfoCommand. 

Unicode encoded metadata text is supported starting from Delphi 2009. It includes:

  • table names;
  • resultset column names;
  • stored procedure names;
  • stored procedure parameter names;
  • SQL command parameter names;
  • SQL command macro names;
  • user name;
  • schema, catalog names.

 

Result set column fields
Driver 
Description 
Advantage Database Server 
  • NCHAR - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • NVARCHAR - ftWideString;
  • NMEMO - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
 
IBM DB2 Server 
  • GRAPHIC - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • VARGRAPHIC - ftWideString;
  • LONG VARGRAPHIC, DBCLOB - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
 
Interbase or Firebird Server
 
If CharacterSet is UTF8 or UNICODE_FSS, then:
  • CHAR - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • VARCHAR - ftWideString;
  • BLOB SUB_TYPE TEXT - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
 
Microsoft SQL Server
 
  • NCHAR - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • NVARCHAR - ftWideString;
  • NTEXT, NVARCHAR(MAX) - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise;
 
MS Access Database 
  • CHARACTER - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • MEMO - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
 
MySQL Server
 
If CharacterSet is UTF8, then:
  • CHAR - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • VARCHAR, TINYTEXT - ftWideString;
  • MEDIUMTEXT, TEXT, LONGTEXT - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
 
Oracle Server
 
  • NCHAR - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • NVARCHAR2 - ftWideString;
  • NCLOB - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
If CharacterSet is UTF8, then:
  • CHAR - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • VARCHAR2 - ftWideString;
  • LONG, CLOB - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
 
SQLite Database
 
If StringFormat=Choose, then:
  • NCHAR, NATIONAL CHAR, NATIONAL CHARACTER - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • NVARCHAR, NVARCHAR2, NATIONAL CHARACTER VARYING, NATIONAL CHAR VARYING - ftWideString;
  • NTEXT, WTEXT, NCLOB, NMEMO, LONG NTEXT, LONG WTEXT, NATIONAL TEXT, LONGWVARCHAR - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
If StringFormat=Unicode, then all string column are Wide strings. If StringFormat=Ansi, then - ANSI strings. 
Postgre SQL
 
If CharacterSet is UTF8, then:
  • CHAR - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • VARCHAR - ftWideString;
  • TEXT - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
 
Sybase SQL Anywhere
 
  • NCHAR - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • NVARCHAR - ftWideString;
  • LONG NVARCHAR - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
 
dbExpress bridge driver
 
Depends on dbExpress driver and DBMS. The general rule:
DBX1-3:
  • fldZSTRING, fldstUNICODE - ftWideString;
  • fldZSTRING, fldstUNICODE, fldstFIXED - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • fldBLOB, fldstMEMO, fldstUNICODE - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise;
  • fldBLOB, fldstFMTMEMO - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
DBX4:
  • TDBXDataTypes.WideStringType - ftWideString;
  • TDBXDataTypes.WideStringType, TDBXDataTypes.FixedSubType - ftFixedWideChar;
  • TDBXDataTypes.BlobType, TDBXDataTypes.WideMemoSubType - ftWideMemo.
 
ODBC bridge driver
 
Depends on ODBC driver and DBMS. The general rule:
  • SQL_WCHAR, SQL_GRAPHIC - ftFixedWideChar on Delphi 2006 and higher, ftWideString otherwise;
  • SQL_WVARCHAR, SQL_VARGRAPHIC, SQL_LONGVARGRAPHIC - ftWideString;
  • SQL_WLONGVARCHAR, SQL_DBCLOB - ftWideMemo on Delphi 2006 and higher, ftFmtMemo otherwise.
 

 

To read/write Unicode string values programmatically use:

  • TField.AsWideString or TField.Value properties;
  • TField.AsString on Delphi 2009 and higher. On earlier Delphi versions reading / writing of AsString property may lead to conversion loss, since AsString returns ANSI strings there.

To read/write the widestring field on Delphi 2009: 

 

ADQuery1.Edit;
ADQuery1.Fields[0].AsWideString := ADQuery1.Fields[0].AsWideString + 'русский + english';
ADQuery1.Post;

 

To read/write widememo field on Delphi 2009: 

 

ADQuery1.Edit;
ADQuery1.FieldsByName('memo').Assign(Memo1.Lines);
ADQuery1.Post;

 

Parameter values

The Unicode encoded parameter value is converted to the character set supported by the DBMS Unicode and sent to the DBMS. This is not dependent on specified or default client character sets or Delphi versions. 

Firebird and Interbase cannot send Unicode character data to a server if the CharacterSet parameter is not UTF8. 

To read/write parameter value in Delphi 2009 do: 

 

ADQuery1.Params[0].AsWideString := 'русский + english';
// the same as following
ADQuery1.Params[0].Value := 'русский + english';
ADQuery1.Params[0].DataType := ftWideString;

 

Text data files

Starting from Delphi 2009, TADDataMove can read/write Unicode encoded text data files without conversion loss. On prior Delphi versions TADDataMove can read/write Unicode encoded text data files, but internally handles all data as ACP encoded. So, conversion loss is possible. 

To control encoding of:

  • text data file, use TADDataMove.TextFileEncoding property;
  • log file, use TADScript.ScriptOptions.LogFileEncoding property.

To read Unicode data file and create ACP encoded log file do: 

 

ADDataMove1.TextFileName := 'u:\data.txt';
ADDataMove1.TextFileEncoding := enUTF8;
ADDataMove1.LogFileName := 'u:\log.txt';
ADDataMove1.TextFileEncoding := enANSI;
ADDataMove1.Destination := ADQuery1;
ADDataMove1.SourceKind := skText;
ADDataMove1.Execute;

 

Trace output

To enable Unicode encoding for trace file output (MonitorBy=FlatFile), set TADMoniFlatFileClientLink.FileEncoding property value to ecUTF8 or ecUTF16. 

 

Tools and Unicode

All AnyDAC utilities are compiled with Delphi XE and are 100% Unicode enabled.

What do you think about this topic? Send feedback!