plugin(logs): support environment variable resolution for Loki and Splunk

[michaelalang]

Changes

• support environment variable resolution for UID keys in pipelineRunUIDKeyEnv, taskRunUIDKeyEnv
• streamlined UID key resolution for Loki and Splunk by using getLogRequestParams
• added startTime and endTime for Splunk queries
• Handle error cases for invalid record types

Release Notes

Yes, Users can now configure pipelineRunUIDKeyEnv, taskRunUIDKeyEnv to match their LogStore attributes.

If so, briefly describe them here so we can include this description in the
release notes for the next release!

For pull requests with a release note:

* Implements https://github.com/tektoncd/results/issues/1179 
* prepares OTEL semantic conventions https://opentelemetry.io/docs/concepts/semantic-conventions/ by making the lookup customizeable 
* Extends the documented environment variables from https://github.com/tektoncd/results/blob/main/docs/logging-support.md 

Common Configuration
These are the common configuration options for all third party logging APIs.

LOGS_API: Set to true to enable the Logs API.
LOGGING_PLUGIN_API_URL: The URL of the third party logging API.
LOGGING_PLUGIN_TOKEN_PATH: The path to the file containing the API token. (optional)
LOGGING_PLUGIN_NAMESPACE_KEY: The key to use for the namespace filtering.
LOGGING_PLUGIN_CONTAINER_KEY: The key to use for adding container name to the logs. (optional)
LOGGING_PLUGIN_STATIC_LABELS: The static labels to use for the logs.
LOGGING_PLUGIN_PROXY_PATH: The path to the proxy to use for the third party logging API. (optional)
LOGGING_PLUGIN_CA_CERT: The CA certificate to use for the third party logging API. This should ideally be passed as environment variable in the deployment spec of the results-api pod. (optional)
LOGGING_PLUGIN_TLS_VERIFICATION_DISABLE: Set to true to disable TLS verification for the third party logging API. (optional)
LOGGING_PLUGIN_FORWARDER_DELAY_DURATION: This is the max duration in minutes taken by third party logging system to forward and store the logs after completion of taskrun and pipelinerun. This is used to search between start time of runs and completion plus buffer duration.
LOGGING_PLUGIN_QUERY_LIMIT: Sets the query limit for Third Party Logging API if logging backend has a limit on number of log lines returned.
LOGGING_PLUGIN_QUERY_PARAMS: Sets the query params for Third Party Logging API, these can be direction/sort order.Specify them in this format: "foo=bar&direction=backward"
LOGGING_PLUGIN_MULTIPART_REGEX: Sets a Regex for matching parts of the same log. Some log backends (e.g S3) store objects immutably, once stored, you can't append. For long running TaskRun steps, it's not effective to keep such log in memory until the step completes. Instead one can store the log in multiple parts with a name suffix (e.g -1743932245 seconds since the Epoch) and set a regex to match the parts of the same log (e.g -\d{10}$). (optional)
PIPELINERUN_UID_KEY: sets the attribute to be searched for defaults to `kubernetes.labels.tekton_dev_pipelineRunUID`
TASKRUN_UID_KEY: sets the attribute to be searched for defaults to `kubernetes.labels.tekton_dev_taskRunUID`

# additional note for reviewers

Since I am not the best at writing go code, the code snippets are AI powered. I reviewed them and they seem to be adequate. 

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: michaelalang / name: Michaela Lang (c7bee96)

[tekton-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign vdemeester after the PR has been reviewed.
You can assign the PR to them by writing /assign @vdemeester in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[enarha]

/ok-to-test