Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ DBInterface.execute(stmt, ("alice", Date(2026, 1, 2)))
DBInterface.executemany(stmt, (name=["bob", "carol"], joined=[Date(2026, 1, 3), Date(2026, 1, 4)]))
DBInterface.close!(stmt)
cursor = DBInterface.execute(conn, "SELECT * FROM users WHERE id = ?", (17,)) # one-shot prepare + execute
# named parameters bind from a NamedTuple or Dict by name
cursor = DBInterface.execute(conn, "SELECT * FROM users WHERE id = :id OR parent = :id", (id=17,))
DBInterface.lastrowid(DBInterface.execute(conn, "INSERT INTO users (name) VALUES ('dave')"))

# transactions, streaming, bulk loading
Expand Down
44 changes: 38 additions & 6 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ There are two ways to run SQL:
* `DBInterface.execute(conn, sql)` runs a statement over the text protocol.
* `stmt = DBInterface.prepare(conn, sql); DBInterface.execute(stmt, params)` prepares a
statement once and executes it any number of times with parameters bound to its `?`
markers over the binary protocol. `DBInterface.execute(conn, sql, params)` is the
one-shot form (prepare, execute, close).
or `:name` markers over the binary protocol. `DBInterface.execute(conn, sql, params)` is
the one-shot form (prepare, execute, close).

Both return a `MySQL.Cursor`, a [Tables.jl](https://juliadata.github.io/Tables.jl/stable/)
row source, so a result can be materialized as `DataFrame(cursor)`, `Tables.columntable(cursor)`,
Expand All @@ -60,10 +60,9 @@ Parameters are bound by their Julia type: integers, floats, strings (`String`,
`missing`/`nothing` for NULL.

Pass a tuple, named tuple, vector, or `Tables.AbstractRow` for multiple parameters.
Values bind to `?` markers in iteration order; names do not change that order. Named SQL
markers such as `:name` are not supported. A bare scalar binds one parameter; use
`(bytes,)` to bind a byte vector as one binary value. `executemany` takes one collection
per parameter, as in the example below.
Values bind to `?` markers in iteration order; names do not change that order. A bare
scalar binds one parameter; use `(bytes,)` to bind a byte vector as one binary value.
`executemany` takes one collection per parameter, as in the example below.

```julia
stmt = DBInterface.prepare(conn, "INSERT INTO users (name, joined) VALUES (?, ?)")
Expand All @@ -72,6 +71,38 @@ DBInterface.executemany(stmt, (name=["bob", "carol"], joined=[Date(2026, 1, 3),
DBInterface.close!(stmt)
```

#### Named parameters

Write `:name` in place of `?` to bind values by name. MySQL's prepared statements accept only
`?`, so MySQL.jl sends each `:name` to the server as `?` and looks up the values by name when
the statement executes. Pass a `NamedTuple`, or an `AbstractDict` keyed by `Symbol` or
`String`:

```julia
stmt = DBInterface.prepare(conn, "UPDATE users SET name = :name WHERE id = :id OR parent = :id")
DBInterface.execute(stmt, (id=7, name="alice"))
DBInterface.execute(stmt, Dict("id" => 8, "name" => "bob"))
DBInterface.executemany(stmt, (name=["carol", "dan"], id=[9, 10])) # each row binds by name
DBInterface.close!(stmt)
DBInterface.execute(conn, "SELECT * FROM users WHERE id = :id", (id=7,))
```

* A name starts with a letter or `_` and continues with letters, digits, and `_`. Names
are case-sensitive: `:userId` takes the value of `userId`, not `userid`.
* A name used more than once binds the same value each time.
* Keys that the statement does not use are ignored. A missing name raises
`MySQL.MySQLInterfaceError` before anything is sent to the server.
* A statement uses `?` markers or `:name` markers, not both. A tuple, vector, or bare
scalar binds one value per `:name` marker, in the order the markers appear. A row
(`Tables.AbstractRow`) raises an error, because its column order need not match the
markers; bind it by name with `NamedTuple(row)`.
* A `:name` inside a string literal, a quoted identifier, or a comment is not a parameter.
Executable comments (`/*! ... */`) are sent unchanged, and `:=` stays an assignment.
String literals end where the session's `NO_BACKSLASH_ESCAPES` mode says they do when the
statement is prepared.
* When the server does not read a `:name` as a parameter, as with a label in a MariaDB
stored procedure (`lbl:LOOP`), MySQL.jl prepares the SQL as written.

`DBInterface.transaction(conn) do ... end` wraps the block in `START TRANSACTION`/`COMMIT`
(`ROLLBACK` on an exception). If a transaction is already active, including one started
with SQL, the block uses a savepoint. Nested success leaves the outer transaction open;
Expand Down Expand Up @@ -280,6 +311,7 @@ MySQL.ConnectOptions
DBInterface.connect
DBInterface.close!
DBInterface.execute
DBInterface.executemany
DBInterface.executemultiple
DBInterface.prepare
DBInterface.transaction
Expand Down
1 change: 1 addition & 0 deletions src/MySQL.jl
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ include("reaper.jl")
include("connect.jl")
include("connection.jl")
include("cursor.jl")
include("named_parameters.jl")
include("statement.jl")
include("load.jl")
include("precompile.jl")
Expand Down
1 change: 1 addition & 0 deletions src/Protocol/constants.jl
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ const MYSQL_OPTION_MULTI_STATEMENTS_OFF = 0x0001

# Server error codes the client must recognise.
const ER_ACCESS_DENIED_ERROR = 1045
const ER_PARSE_ERROR = 1064
const ER_NET_PACKET_TOO_LARGE = 1153
const ER_SP_DOES_NOT_EXIST = 1305
const ER_QUERY_INTERRUPTED = 1317
Expand Down
129 changes: 129 additions & 0 deletions src/named_parameters.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# Named parameters for prepared statements. The server's prepared statements take positional
# `?` markers only, so `DBInterface.prepare` rewrites each `:name` to `?` and keeps the names,
# and `DBInterface.execute` looks each value up by name in a NamedTuple or AbstractDict.

"""
MySQL.NamedSQL

The SQL of a statement with named parameters as the server prepares it: `sql` has each
`:name` replaced by `?`, and `names[i]` is the name of the `i`-th `?` (a reused name repeats).
"""
struct NamedSQL
sql::String
names::Vector{Symbol}
end

is_name_start(b::UInt8) = return (UInt8('a') <= b <= UInt8('z')) | (UInt8('A') <= b <= UInt8('Z')) | (b == UInt8('_')) | (b >= 0x80)
is_name_char(b::UInt8) = return is_name_start(b) | (UInt8('0') <= b <= UInt8('9'))

"""
MySQL.named_sql(sql, no_backslash_escapes) -> Union{Nothing, NamedSQL}

Finds the `:name` parameters of `sql` the way the server's lexer reads the text, and returns
the rewritten SQL, or `nothing` when `sql` has none. A name starts with a letter, `_`, or a
non-ASCII character, and continues with those or digits; `:=` and `:1` are not parameters.
String literals (`'...'` and `"..."`, where a backslash escapes the next byte unless
`no_backslash_escapes`), backtick identifiers, and comments (`#`, `-- `, `/* */`) are skipped.
Executable comments (`/*! */`, `/*M! */`) are skipped too: the server runs or ignores them
depending on its version, so their text is never changed.
"""
function named_sql(sql::AbstractString, no_backslash_escapes::Bool)
# Positional SQL usually has no colon at all: one memchr.
findfirst(==(UInt8(':')), codeunits(sql)) === nothing && return nothing
str = String(sql)
bytes = codeunits(str)
span = next_name(bytes, 1, no_backslash_escapes)
span === nothing && return nothing
names = Symbol[]
out = UInt8[]
copied = 0 # bytes[1:copied] are already in `out`
while span !== nothing
colon, stop = span
push!(names, Symbol(SubString(str, colon + 1, prevind(str, stop + 1))))
append!(out, view(bytes, (copied + 1):(colon - 1)))
push!(out, UInt8('?'))
copied = stop
span = next_name(bytes, stop + 1, no_backslash_escapes)
end
append!(out, view(bytes, (copied + 1):length(bytes)))
return NamedSQL(String(out), names)
end

# `(colon, stop)`: the byte range of the next `:name` at or after `i`, or `nothing`.
function next_name(bytes::Base.CodeUnits{UInt8, String}, i::Int, no_backslash_escapes::Bool)
n = length(bytes)
while i <= n
b = bytes[i]
if b == UInt8('\'') || b == UInt8('"')
i = skip_quoted(bytes, i, !no_backslash_escapes)
elseif b == UInt8('`')
i = skip_quoted(bytes, i, false)
elseif b == UInt8('#') || (b == UInt8('-') && i < n && bytes[i + 1] == UInt8('-') &&
(i + 2 > n || bytes[i + 2] <= 0x20 || bytes[i + 2] == 0x7f))
newline = findnext(==(UInt8('\n')), bytes, i)
i = newline === nothing ? n + 1 : newline + 1
elseif b == UInt8('/') && i < n && bytes[i + 1] == UInt8('*')
i += 2
while i < n && !(bytes[i] == UInt8('*') && bytes[i + 1] == UInt8('/'))
i += 1
end
i += 2
elseif b == UInt8(':') && i < n && is_name_start(bytes[i + 1])
stop = i + 1
while stop < n && is_name_char(bytes[stop + 1])
stop += 1
end
return (i, stop)
else
i += 1
end
end
return nothing
end

# Index just past the quoted run that starts at `bytes[i]`. A doubled quote (`''`) ends one
# run and starts the next, which skips the same bytes.
function skip_quoted(bytes::Base.CodeUnits{UInt8, String}, i::Int, backslash_escapes::Bool)
q = bytes[i]
i += 1
while i <= length(bytes)
b = bytes[i]
b == q && return i + 1
i += (backslash_escapes && b == UInt8('\\')) ? 2 : 1
end
return i
end

# The values of a named statement's `?` markers, looked up by name in a NamedTuple or
# AbstractDict; the positional encoder iterates it like a tuple.
struct NamedValues{T}
names::Vector{Symbol}
params::T
end

Base.length(v::NamedValues) = return length(v.names)
Base.getindex(v::NamedValues, i::Int) = return param_value(v.params, v.names[i])
Base.iterate(v::NamedValues, i::Int=1) = return i > length(v.names) ? nothing : (v[i], i + 1)

has_param(p::NamedTuple, name::Symbol) = return haskey(p, name)
param_value(p::NamedTuple, name::Symbol) = return getfield(p, name)
has_param(p::AbstractDict, name::Symbol) = return haskey(p, name) || haskey(p, String(name))
param_value(p::AbstractDict, name::Symbol) = return haskey(p, name) ? p[name] : p[String(name)]

function named_values(names::Vector{Symbol}, params::Union{NamedTuple, AbstractDict})
for name in names
has_param(params, name) || missing_param(name)
end
return NamedValues(names, params)
end

# A tuple or vector binds one value per marker in marker order, the way MariaDB binds its own
# `:name` markers under sql_mode=ORACLE.
named_values(names::Vector{Symbol}, params::Union{Tuple, AbstractVector}) = return params

# A row has names but iterates in column order, which need not match the markers.
named_values(names::Vector{Symbol}, params) = throw(MySQLInterfaceError(
"a statement with named parameters takes a NamedTuple or AbstractDict (bound by name) or a tuple or vector (bound in marker order), not a $(typeof(params))",
))

@noinline missing_param(name::Symbol) = throw(MySQLInterfaceError("no value for named parameter :$name"))
106 changes: 91 additions & 15 deletions src/statement.jl
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,14 @@ end

A prepared statement on the native backend, from `DBInterface.prepare(conn, sql)`. Execute it
with `DBInterface.execute(stmt, params)`; close it with `DBInterface.close!(stmt)` (the
COM_STMT_CLOSE is deferred to the next command, never sent from a finalizer).
COM_STMT_CLOSE is deferred to the next command, never sent from a finalizer). `stmt.sql` is
the SQL as written, `:name` parameters included.
"""
mutable struct Statement <: DBInterface.Statement
conn::Connection
statement_id::UInt32
sql::String
named::Union{Nothing, NamedSQL} # `nothing`: no `:name` parameters, `sql` is prepared as written
generation::Int
nparams::Int
params::Vector{P.ColumnDef}
Expand Down Expand Up @@ -67,19 +69,31 @@ end
"""
DBInterface.prepare(conn::MySQL.Connection, sql) -> Statement

Prepares `sql` on the server and returns a `Statement`.
Prepares `sql` on the server and returns a `Statement`. Parameters are positional `?` markers
or named `:name` markers; one statement cannot mix the two. The server takes only `?`, so each
`:name` is sent as `?` and execution binds its value by name. A `:name` inside a string
literal, a quoted identifier, or a comment (executable `/*! */` comments included) is left
alone. When the server does not read every `:name` as a parameter, as with a MariaDB
stored-program label such as `lbl:LOOP`, the SQL is prepared as written and binds positionally.
"""
function DBInterface.prepare(conn::Connection, sql::AbstractString)
lock(conn.lock) do
s = begin_command!(conn)
P.stmt_prepare!(s, sql)
ok = P.read_prepare_response!(s)
named = named_sql(sql, (s.status & P.SERVER_STATUS_NO_BACKSLASH_ESCAPES) != 0)
ok = if named === nothing
P.stmt_prepare!(s, sql)
P.read_prepare_response!(s)
else
ok, named = prepare_named!(s, sql, named)
ok
end
names, types, lookup, coltypes = statement_schema(conn, ok)
generation = @atomic conn.generation
stmt = Statement(
conn,
ok.statement_id,
String(sql),
named,
generation,
P.num_params(ok),
ok.params,
Expand All @@ -101,13 +115,40 @@ function DBInterface.prepare(conn::Connection, sql::AbstractString)
end
end

# Re-prepares `stmt.sql` on the current (READY) session and refreshes its id/generation and
# cached metadata. A 1615 retry closes the superseded id on the same session. A reconnect
# leaves the old-generation id alone because it belongs to the dead session.
# Prepares the `?` rewrite. If the server does not read every `:name` as a parameter (the
# rewrite does not parse, or it counts other parameters), the SQL is prepared as written:
# MariaDB prepares stored-program labels (`lbl:LOOP`) and Oracle-mode `:NEW.col`, which have no
# `?` form. If that fails too, the rewrite's problem is reported.
function prepare_named!(s::P.Session, sql::AbstractString, named::NamedSQL)
P.stmt_prepare!(s, named.sql)
problem = try
ok = P.read_prepare_response!(s)
P.num_params(ok) == length(named.names) && return ok, named
P.stmt_close!(s, ok.statement_id)
MySQLInterfaceError("the server found $(P.num_params(ok)) parameters but the statement names $(length(named.names)); a statement cannot mix `?` and `:name` parameters")
catch err
(err isa P.StmtError && err.errno == P.ER_PARSE_ERROR) || rethrow()
err
end
P.stmt_prepare!(s, sql)
ok = try
P.read_prepare_response!(s)
catch err
err isa P.StmtError || rethrow()
throw(problem)
end
return ok, nothing
end

# Re-prepares the statement (its `?` rewrite when it has `:name` parameters) on the current
# (READY) session and refreshes its id/generation and cached metadata. A 1615 retry closes the
# superseded id on the same session. A reconnect leaves the old-generation id alone because it
# belongs to the dead session.
function reprepare!(conn::Connection, s::P.Session, stmt::Statement; close_previous::Bool=false, replay_long_data::Bool=true)
old_id = stmt.statement_id
old_generation = stmt.generation
P.stmt_prepare!(s, stmt.sql)
named = stmt.named
P.stmt_prepare!(s, named === nothing ? stmt.sql : named.sql)
ok = P.read_prepare_response!(s)
generation = @atomic conn.generation
if close_previous && old_generation == generation && old_id != ok.statement_id
Expand Down Expand Up @@ -282,16 +323,28 @@ normalize_params(p::Union{Number, AbstractString, Missing, Nothing, Dates.TimeTy
"""
DBInterface.execute(stmt::MySQL.Statement, params=(); mysql_store_result=true) -> BinaryCursor

Executes the prepared statement with `params` bound as the `?` markers and returns a
binary-protocol cursor. A tuple, named tuple, vector, or `Tables.AbstractRow` supplies values
in iteration order; names do not select SQL parameters. A bare scalar binds one parameter.
Wrap a binary byte vector as `(bytes,)` to bind it as one value. Named SQL markers such as
`:name` are not supported. `mysql_store_result=false` streams rows (the connection is busy until
the cursor is exhausted or closed).
Executes the prepared statement with `params` bound to its parameters and returns a
binary-protocol cursor. For `?` markers, a tuple, named tuple, vector, or `Tables.AbstractRow`
supplies values in iteration order; names do not select SQL parameters. A bare scalar binds one
parameter. Wrap a binary byte vector as `(bytes,)` to bind it as one value. For `:name` markers,
pass a `NamedTuple` or an `AbstractDict` keyed by `Symbol` or `String`: each marker takes the
value of its name (case-sensitive), a reused name binds the same value, and keys the SQL does
not name are ignored. A missing name raises an error before anything is sent. A tuple, vector,
or bare scalar binds one value per `:name` marker in marker order; a `Tables.AbstractRow` raises
an error (convert it with `NamedTuple(row)` to bind by name). `mysql_store_result=false`
streams rows (the connection is busy until the cursor is exhausted or closed).
"""
function DBInterface.execute(stmt::Statement, params=(); mysql_store_result::Bool=true)
conn = stmt.conn
params = normalize_params(params)
named = stmt.named
# Two calls, not one variable: a Union of both value types would box the parameters.
named === nothing && return execute_values(stmt, params, mysql_store_result)
return execute_values(stmt, named_values(named.names, params), mysql_store_result)
end

# Executes with `params` in `?` marker order.
function execute_values(stmt::Statement, params, mysql_store_result::Bool)
conn = stmt.conn
lock(conn.lock) do
stmt.closed && closed_statement()
check_paramcount(stmt, params)
Expand Down Expand Up @@ -351,6 +404,29 @@ function DBInterface.executemultiple(stmt::Statement, params=(); mysql_store_res
return Cursors(stmt.conn, stmt.sql, first.opts, first)
end

"""
DBInterface.executemany(stmt::MySQL.Statement, params::NamedTuple)

Executes `stmt` once per row of a batch that holds one equal-length collection per parameter,
inside one transaction. For `:name` markers each row binds by name; for `?` markers each row
binds in field order, as in DBInterface's generic method.
"""
function DBInterface.executemany(stmt::Statement, params::NamedTuple)
# DBInterface hands each row to `execute` positionally, which a `:name` statement rejects.
(stmt.named === nothing || isempty(params)) &&
return invoke(DBInterface.executemany, Tuple{DBInterface.Statement, Any}, stmt, params)
n = length(first(params))
all(c -> length(c) == n, params) ||
throw(DBInterface.ParameterError("parameter collections provided to `DBInterface.executemany` must have equal lengths"))
n == 0 && return nothing
DBInterface.transaction(stmt.conn) do
for i in 1:n
DBInterface.close!(DBInterface.execute(stmt, map(c -> c[i], params)))
end
end
return nothing
end

"""
DBInterface.close!(stmt::MySQL.Statement)

Expand Down
Loading
Loading