6 Notes

▼ |  | DB: Database Connectivity

1 | Using Database Connections
2 | Connections
3 | Queries
4 | SQL Types and Conversions
5 | Utilities
6 | Notes

► | 6 | Notes

6.1 | Local Sockets for Postgre SQL and My SQL Servers
6.2 | Postgre SQL Database Character Encoding
6.3 | Postgre SQL Authentication
6.4 | Postgre SQL Timestamps and Time Zones
6.5 | My SQL Authentication
6.6 | My SQL Connection Character Set
6.7 | My SQL CALLing Stored Procedures
6.8 | Cassandra Authentication
6.9 | SQLite Requirements
6.10 | FFI-Based Connections and Concurrency
6.11 | ODBC Requirements
6.12 | ODBC Status
6.13 | Multi-Statement Queries

On this page:

6.1 Local Sockets for Postgre SQL and My SQL Servers
6.2 Postgre SQL Database Character Encoding
6.3 Postgre SQL Authentication
6.4 Postgre SQL Timestamps and Time Zones
6.5 My SQL Authentication
6.6 My SQL Connection Character Set
6.7 My SQL CALLing Stored Procedures
6.8 Cassandra Authentication
6.9 SQLite Requirements
6.10 FFI-Based Connections and Concurrency
6.11 ODBC Requirements
6.12 ODBC Status
6.12.1 DB2 ODBC Driver
6.12.2 Oracle ODBC Driver
6.12.3 SQL Server ODBC Driver
6.13 Multi-Statement Queries

top contents ← prev up next →

## 6 Notes🔗ℹ

This section discusses issues related to specific database systems.

### 6.1 Local Sockets for PostgreSQL and MySQL Servers🔗ℹ

PostgreSQL and MySQL servers are sometimes configured by default to listen only on local sockets (also called “unix domain sockets”). This library provides support for communication over local sockets on Linux and Mac OS. If local socket communication is not available, the server must be reconfigured to listen on a TCP port.

The socket file for a PostgreSQL server is located in the directory specified by the unix_socket_directory variable in the postgresql.conf server configuration file. For example, on Ubuntu 11.04 running PostgreSQL 8.4, the socket directory is /var/run/postgresql and the socket file is /var/run/postgresql/.s.PGSQL.5432. Common socket paths may be searched automatically using the postgresql-guess-socket-path function.

The socket file for a MySQL server is located at the path specified by the socket variable in the my.cnf configuration file. For example, on Ubuntu 11.04 running MySQL 5.1, the socket is located at /var/run/mysqld/mysqld.sock. Common socket paths for MySQL can be searched using the mysql-guess-socket-path function.

### 6.2 PostgreSQL Database Character Encoding🔗ℹ

In most cases, a database’s character encoding is irrelevant, since the connect function always requests translation to Unicode (UTF-8) when creating a connection. If a PostgreSQL database’s character encoding is SQL_ASCII, however, PostgreSQL will not honor the connection encoding; it will instead send untranslated octets, which will cause corrupt data or internal errors in the client connection.

To convert a PostgreSQL database from SQL_ASCII to something sensible, pg_dump the database, recode the dump file (using a utility such as iconv), create a new database with the desired encoding, and pg_restore from the recoded dump file.

### 6.3 PostgreSQL Authentication🔗ℹ

PostgreSQL supports a large variety of authentication mechanisms, controlled by the pg_hba.conf server configuration file. This library currently works with the following authentication methods:

- peer: only for local sockets

- password, ldap, pam, radius, bsd: cleartext password, only if explicitly allowed (see postgresql-connect)

- md5: MD5-hashed password

- scram-sha-256: password-based challenge/response protocol using SASL

  See SCRAM-SHA-256 and SCRAM-SHA-256-PLUS from RFC 7677.

- oauth: OAuth 2.0 bearer token authentication using SASL. This library does not directly support any authorization flows; tokens must be obtained by other means.

  See OAUTHBEARER from RFC 7628.

The gss, sspi, and krb5 methods are not supported.

Changed in version 1.2 of package db-lib: Added SCRAM-SHA-256 support.
Changed in version 1.7: Added SCRAM-SHA-256-PLUS support.
Changed in version 1.12: Added OAUTHBEARER support.

### 6.4 PostgreSQL Timestamps and Time Zones🔗ℹ

PostgreSQL’s timestamp with time zone type is inconsistent with the SQL standard (probably), inconsistent with time with time zone, and potentially confusing to PostgreSQL newcomers.

A time with time zone is essentially a time structure with an additional field storing a time zone offset. In contrast, a timestamp with time zone has no fields beyond those of timestamp. Rather, it indicates that its datetime fields should be interpreted as a UTC time. Thus it represents an absolute point in time, unlike timestamp without time zone, which represents local date and time in some unknown time zone (possibly—hopefully—known the the database designer, but unknown to PostgreSQL).

When a timestamp with time zone is created from a source without time zone information, the session’s TIME ZONE setting is used to adjust the source to UTC time. When the source contains time zone information, it is used to adjust the timestamp to UTC time. In either case, the time zone information is discarded and only the UTC timestamp is stored. When a timestamp with time zone is rendered as text, it is first adjusted to the time zone specified by the TIME ZONE setting (or by AT TIME ZONE) and that offset is included in the rendered text.

This library receives timestamps in binary format, so the time zone adjustment is not applied, nor is the session’s TIME ZONE offset included; thus all sql-timestamp values in a query result have a tz field of 0 (for timestamp with time zone) or #f (for timestamp without time zone). (Previous versions of this library sent and received timestamps as text, so they received timestamps with adjusted time zones.)

### 6.5 MySQL Authentication🔗ℹ

As of version 5.5.7, MySQL supports authentication plugins. This library supports the following plugins:

- caching_sha2_password: the default since MySQL version 8.0

- mysql_native_password: the default for MySQL versions since 4.1 and before 8.0

- mysql_old_password: the default before MySQL version 4.1

- mysql_clear_password: used by LDAP and PAM authentication

The caching_sha2_password authentication plugin has two “paths”; a client always tries the fast path first, but the server may demand that it go through the slow path, based on the state of the server’s authentication cache. The fast path uses a challenge-response protocol. The slow path is divided into the following cases:

- connection via unix socket or via TCP with TLS to localhost: The client simply sends the password to the server.

- connection via TCP with TLS, but not to localhost: The client sends the password to the server if the allow-cleartext-password? argument is true; otherwise, an exception is raised.

- connection via TCP without TLS: Not supported by this library; an exception is raised.

See also Making Database Connections Securely.

Changed in version 1.6 of package db-lib: Added support for caching_sha2_password authentication.

### 6.6 MySQL Connection Character Set🔗ℹ

This library communicates with MySQL servers using UTF-8 for all character data. MySQL has two different UTF-8 character sets: utf8 (sometimes called utf8mb3) is a nonstandard version limited to three bytes, and utf8mb4 is standard UTF-8. Each character set has multiple collations, and the available collations and the default collation may vary based on server version. This library initializes a connection’s character set and collation as follows:

- if the collation in the server handshake is either utf8mb4_general_ci or utf8mb4_0900_ai_ci, then the connection uses that collation, with character set utf8mb4;

- if the server version is at least 5.5.3, the connection uses collation utf8mb4_general_ci, with character set utf8mb4; otherwise

- the connection uses collation utf8_general_ci, with character set utf8 (utf8mb3).

Previous versions of this library issued a SET NAMES utf8 command at the beginning of every connection.

Warning: If the client, connection, or result character sets are changed (for example, using SET NAMES) to a character set other than UTF-8, errors or data corruption may occur. Note that non-UTF-8 character sets attached to databases, tables, and columns do not cause problems; the server automatically translates between character set used for storage and the one used for communication.

### 6.7 MySQL CALLing Stored Procedures🔗ℹ

MySQL CALL statements can be executed only if they return at most one result set and contain no OUT or INOUT parameters.

### 6.8 Cassandra Authentication🔗ℹ

Cassandra, like MySQL, supports authentication plugins. The only plugins currently supported by this library are AllowAllAuthenticator and PasswordAuthenticator.

### 6.9 SQLite Requirements🔗ℹ

SQLite support requires the appropriate native library.

- On Windows, the library is sqlite3.dll. It is included in the Racket distribution.

- On Mac OS, the library is libsqlite3.0.dylib, which is included (in /usr/lib) in Mac OS version 10.4 onwards.

- On Linux, the library is libsqlite3.so.0. It is included in the libsqlite3-0 package in Debian/Ubuntu and in the sqlite package in Red Hat.

### 6.10 FFI-Based Connections and Concurrency🔗ℹ

Wire-based connections communicate using ports, which do not cause other Racket threads to block. In contrast, an FFI call causes all Racket threads to block until it completes, so FFI-based connections can degrade the interactivity of a Racket program, particularly if long-running queries are performed using the connection.

This problem can be avoided by creating the FFI-based connection in a separate place using the #:use-place keyword argument. Such a connection will not block all Racket threads during queries; the disadvantage is the cost of creating and communicating with a separate place. On Racket CS, another solution is to execute queries in a separate OS thread; this solution may have lower time and memory overhead than the separate place.

### 6.11 ODBC Requirements🔗ℹ

ODBC requires the appropriate driver manager native library as well as driver native libraries for each database system you want use ODBC to connect to.

- On Windows, the driver manager is odbc32.dll, which is included automatically with Windows.

- On Mac OS, the driver manager is libiodbc.2.dylib (iODBC), which is included (in /usr/lib) in Mac OS version 10.2 onwards.

- On Linux, the driver manager is libodbc.so.{2,1} (unixODBC—iODBC is not supported). It is available from the unixodbc package in Debian/Ubuntu and in the unixODBC package in Red Hat.

In addition, you must install the appropriate ODBC Drivers and configure Data Sources. Refer to the ODBC documentation for the specific database system for more information.

### 6.12 ODBC Status🔗ℹ

ODBC support is experimental. The behavior of ODBC connections can vary widely depending on the driver in use and even the configuration of a particular data source.

The following sections describe the configurations that this library has been tested with. Reports of success or failure on other platforms or with other drivers would be appreciated.

#### 6.12.1 DB2 ODBC Driver🔗ℹ

IBM DB2 ODBC drivers were tested with the following software configuration:

- Platform: Centos 7.4 on x86_64

- Database: DB2 Express-C for Linux x64 v11.1

- Driver: ODBC for DB2 (included with DB2 Express-C)

This driver seems to require environment variables to be set using the provided scripts (eg, source /home/db2inst1/sqllib/db2profile).

Known issues:

- The driver does not support the standard SQL_C_NUMERIC structure for retrieving DECIMAL/NUMERIC fields.

  Fix: Use #:quirks '(no-c-numeric) with odbc-connect.

#### 6.12.2 Oracle ODBC Driver🔗ℹ

Oracle ODBC drivers were tested with the following software configuration:

- Platform: Centos 7.4 on x86_64

- Database: Oracle XE 11g (11.2.0)

- Drivers: Oracle Instant Client ODBC (11.2.0 and 12.2.0)

Typical installations of the drivers require the LD_LIBRARY_PATH environment variable to be set to the driver’s installed lib directory (ie, the directory containing libsqora.so) so the driver can find its sibling shared libraries.

Known issues:

- With the #:strict-parameter-types? #t option, parameters seem to be always assigned the type varchar.

  Fix: Leave strict parameter types off (the default).

- The driver does not support the SQL_C_BIGINT format for parameters or result fields. Consequently, passing large integers as query parameters may fail.

  Fix: Use #:quirks '(no-c-bigint) with odbc-connect.

- A field of type TIME causes the driver to return garbage for the typeid and type parameters. This usually causes an error with a message like “unsupported type; typeid: -29936”, but with a random typeid value. (Oracle appears not to have a TIME type, so this bug might only appear when a value is explicitly CAST as TIME—for some reason, that doesn’t produce an error.)

- Attempting to quit Racket with a connection still open may cause Racket to hang. Specifically, the problem seems to be in the driver’s _fini function.

  Fix: Close connections before exiting, either explicitly using disconnect or by shutting down their custodians.

#### 6.12.3 SQL Server ODBC Driver🔗ℹ

Microsoft SQL Server ODBC drivers were tested with the following software configuration:

- Platform: Windows 10 on x86_64

- Database: SQL Server Express 2017

- Drivers: ODBC Driver 13 for SQL Server, SQL Server Native Client 11.0

Known issues:

- If queries are nested or interleaved—that is, a second query is executed before the first query’s results are completely consumed—the driver might signal an error “Connection is busy with results for another command (SQLSTATE: HY000)”.

  Fix: Set the MARS_Connection data source option to Yes (see this page). The ODBC Manager GUI does not expose the option, but it can be added by editing the registry.

### 6.13 Multi-Statement Queries🔗ℹ

This library does not directly support multi-statement queries. That is, each query operation must be given exactly one top-level SQL statement; otherwise the operation raises an exception. For example, the following query operation is invalid:

> (query c "INSERT INTO t VALUES (1); INSERT INTO t VALUES (2);") ; invalid

Multi-statement queries are not supported because they are not generally supported by the backend-specific wire protocols and APIs that this library is built on.

Workarounds for a few database systems are available:

- PostgreSQL: Inserting, updating, or deleting many rows with a single query is possible using the UNNEST function on array arguments. See for example Postgres UNNEST cheat sheet for bulk operations.

  To execute dissimilar statements as a single query, wrap the statements in a DO statement. Note that the body of a DO statement must be given as a string literal; use PostgreSQL’s dollar-quoted string literals to avoid the need to escape nested string literals. For example:

  > (query c "DO $$BEGIN INSERT INTO t VALUES (1); INSERT INTO t VALUES (2); END$$")

- MySQL, some other systems: Wrap the statements in a new stored procedure using CREATE PROCEDURE, then CALL the procedure to execute the statements, and then DROP the stored procedure. The procedure must cause at most one result set to be returned.

- SQLite: No known workarounds. In particular, SQLite does not support stored procedures.

top contents ← prev up next →