Added JDBC db.query.parameter span attributes#13719
Added JDBC db.query.parameter span attributes#13719laurit merged 21 commits intoopen-telemetry:mainfrom
Conversation
| "SELECT 3", | ||
| "SELECT ?", | ||
| "SELECT 3, $1", | ||
| "SELECT ?, $1", |
There was a problem hiding this comment.
Why is the 3 sanitized in this case? (not related to this PR, just a regular question)
There was a problem hiding this comment.
Sanitizer replaces all literals with ?. It does not distinguish where the literal appears or whether it just a random 3 or something sensitive like a password or a credit card number. The purpose of the sanitizer is to protect the users and apm vendors from accidentally sending sensitive information to the apm. Without having looked too much into your PR In my opinion it might be best if you first focus on prepared statements, if users wishes to see parameter values in sanitized queries they can disable the sanitizer.
There was a problem hiding this comment.
it might be best if you first focus on prepared statements, if users wishes to see parameter values in sanitized queries they can disable the sanitizer
👍
There was a problem hiding this comment.
My concern with this PR is more to see parameters of the prepared statement itself. Currently, if I am doing things right, even with the sanitizer disabled, here is what I get from the library when I wrap with an OpenTelmetryDataSource in one of my app
SELECT pg_advisory_xact_lock(('x' || md5(?))::bit(64)::bigint)where as my initial scala code (Scala + Doobie) looks like this
private def acquireLockByIdsNel(ids: NonEmptyList[String]): Query0[Unit] = {
val unique = ids.distinct.sorted
val frg = unique.map(id => fr"pg_advisory_xact_lock(('x' || md5($id))::bit(64)::bigint)")
val all = fragments.comma(frg)
(fr"SELECT" ++ all).queryWithLabel[Unit]("SELECT locks")
}as this get translated into a PreparedStatement and having no clue about what is currently sent to the DB, I'd like db.(operation|query).parameter to be set. Thing is, ? is already used by the sanitizer. So in order to make things clear for the user, I have to(?) expose them as parameters.
Let me know if I misunderstood or going the wrong direction!
There was a problem hiding this comment.
Thing is, ? is already used by the sanitizer. So in order to make things clear for the user, I have to(?) expose them as parameters.
I wouldn't be too concerned with this. Firstly users can disable the sanitizer, as far as I can tell currently you basically attempt to undo what the sanitizer does by capturing the value. Secondly I'd expect that most queries that use prepared statements don't contain literals that the sanitizer removes. In your place I would start with capturing the parameters for prepared statements.
There was a problem hiding this comment.
Then I think that this PR does exactly what I want!
I can remove the db.query.parameter for sanitized parameters from the query if that's an issue. But other than that, the PR captures the parameters from prepared statements
There was a problem hiding this comment.
I'd expect that most queries that use prepared statements don't contain literals that the sanitizer removes
ah, I think we should remove sanitization of prepared statement text in upcoming stable database semconv
this will also ensure the ? placeholders line up with the db.query.parameter list
I've added this to our tracking issue #12608
|
@AlixBa check out very recent semconv change: open-telemetry/semantic-conventions#2093 |
|
hey @laurit |
| | System property | Type | Default | Description | | ||
| |---------------------------------------------------------|---------|---------|----------------------------------------------------| | ||
| | `otel.instrumentation.jdbc.statement-sanitizer.enabled` | Boolean | `true` | Enables the DB statement sanitization. | | ||
| | `otel.instrumentation.jdbc.query-parameter.enabled` | Boolean | `false` | Enables the attribute db.query.parameter.<key> | |
There was a problem hiding this comment.
This table should be reformatted. The description of the flag needs to include a warning. I'll leave it to @trask to come up with the exact wording, probably something along the lines of WARNING: captured query parameters may contain sensitive information such as passwords, personally identifiable information or protected health info. Exposing such info may result in substantial fines and penalties or criminal liability. Consult your peers, superiors and a legal counsel before enabling this option.
There was a problem hiding this comment.
it's a good question, I've asked in the community repo
| * personally identifiable information or protected health info. Exposing such info may result in | ||
| * substantial fines and penalties or criminal liability. Consult your peers, superiors and a | ||
| * legal counsel before enabling this option. |
There was a problem hiding this comment.
(and elsewhere, based on open-telemetry/community#2713 (comment))
| * personally identifiable information or protected health info. Exposing such info may result in | |
| * substantial fines and penalties or criminal liability. Consult your peers, superiors and a | |
| * legal counsel before enabling this option. | |
| * personally identifiable information or protected health info. |
Contribution for #7413
Not entirely sure about/need help/to finish
What does it do when opted-in?
Stringified types: