Skip to content

Add mechanism to be able query metrics from hypershift clusters - #195

Merged
naved001 merged 2 commits into
CCI-MOC:mainfrom
naved001:hypershift-update
Oct 26, 2026
Merged

naved001 merged 2 commits into
CCI-MOC:mainfrom
naved001:hypershift-update

Conversation

@naved001

@naved001 naved001 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

With clusters deployed using hosted control plane (hypershift) the metrics kube_pod_resource_request now come from the hosted cluster where the query requires an additional parameter. This PR adds a mechanism to pass cluster_id to our queries.

the queries kube_node_label and kube_pod_labels are ignored for now as they are still on the hosted cluster. it's easy to workaround those 2 since we can supply the information of kube_node_labels statically (we have already have a mechanism for that). Support for gathering kube_pod_labels will added in a different PR (we'll either query the other prometheus or wait for multi cluster observibility to have all the data in one place).

This also drops support for kube_pod_status_unschedulable because 1. this will need to come from the hosted cluster and more importantly 2. it is just redundant as we already check for node != "" in our kube_pod_resource_requests query.

Disclaimer: I definitely used my brand new claude pro subscription lol

@naved001
naved001 requested a review from QuanMPhm October 7, 2026 19:31
@QuanMPhm
QuanMPhm requested review from KelvinLinBU and marcoagonzales007 and removed request for KelvinLinBU October 13, 2026 15:57
@QuanMPhm

Copy link
Copy Markdown
Contributor

Added @marcoagonzales007 as a reviewer to give him some exposure to this repo. @marcoagonzales007 Feel free to learn the code and leave a review whenever you have time.

Comment on lines +45 to +48
def test_cluster_id_with_special_characters(self):
cluster_id = "test-cluster-123_xyz"
query = QueryBuilder.build_cpu_request("hypershift", cluster_id)
assert f'_id="{cluster_id}"' in query

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this test nessecary? I suppose there's no harm but I'm not sure what the benefit is.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not necessary, its just slop so I removed it.

Comment thread openshift_metrics/query_builder.py Outdated
Comment on lines +11 to +13
if cluster_type == "hypershift":
base_query += f', _id="{cluster_id}"'
base_query += "}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block is repeated 3 times in this file. Would it make sense to put this in a class function?
Since the script seems to expect to be ran with only a cluster type at a time, would it also make sense to make cluster_type a class argument instead an arguement to each "build_request" function?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point!

With clusters deployed using hosted control plane (hypershift) the
metrics kube_pod_resource_request now come from the hosted cluster where
the query requires an additional parameter. This PR adds a mechanism to
pass cluster_id to our queries.

the queries kube_node_label and kube_pod_labels are ignored for now as
they are still on the hosted cluster. it's easy to workaround those 2
since we can supply the information of kube_node_labels statically (we
have already have a mechanism for that). Support for gathering
kube_pod_labels will added in a different PR (we'll either query the
other prometheus or wait for multi cluster observibility to have all the
data in one place).

This also drops support for kube_pod_status_unschedulable because 1.
this will need to come from the hosted cluster and more importantly 2.
it is just redundant as we already check for node != "" in our
kube_pod_resource_requests query.
Looking at the tests again I wanted them to use more realistic data so I
updated them
@naved001
naved001 requested a review from knikolla October 26, 2026 14:44
@naved001
naved001 merged commit 1118168 into CCI-MOC:main Oct 26, 2026
3 checks passed
@naved001
naved001 deleted the hypershift-update branch October 26, 2026 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants