Provide dedicated error for "Could not connect to lakeFS"

[BigRoy]

Please describe the feature you have in mind and explain what the current shortcomings are?

Currently it's hard to identify whether a resolver timestamp missing on LakeFS or any USD lib items unable to be found on the LakeFS server are because it can't connect to the server or whether the matching resolver doesn't exist on the LakeFS repository. Hence, the logs we generate are unclear regarding whether it failed to connect due to any of:

• missing or invalid lakefs server credentials
• wrong lakefs repo uri
• missing resolver item on the server (this is what it reports now)

How would you imagine the implementation of the feature?

Get a connection status to the repo itself, detect whether it's able to access the server, whether it's able to login or not, etc. and report accordingly. This will make it easier to debug the issue that users might face.

[Child Issues]

• List Repos and Check Connection Function.  ayon-bin-bridge-client#19

[Lypsolon]

There are better options im sure but what we could do is the following.

we first ping the server to see if the url is available.
then we use

lakectl repo list

this gives us the HTTP status code back and is fast enough.

Warning: failed sending statistics: 401 Unauthorized
error authenticating request
401 Unauthorized

[BigRoy]

The status check response is very good - it may be good that if a status code comes back that is not positive that we ALWAYS report that as the exception for clarity? (Unless we know more specifics about the error than the status code, if so we could then of course provide an even better exception)

Anyway, what does lakectl repo list return as status code if the URL is not available? Maybe that already reports that it can't connect to the URL? (Like a "no response" or whatever?)

[Lypsolon]

The status check response is very good - it may be good that if a status code comes back that is not positive that we ALWAYS report that as the exception for clarity? (Unless we know more specifics about the error than the status code, if so we could then of course provide an even better exception)

Anyway, what does lakectl repo list return as status code if the URL is not available? Maybe that already reports that it can't connect to the URL? (Like a "no response" or whatever?)

this is what it returns when the URL is wrong.

➜  ~ lakectl repo list 
Warning: failed sending statistics: Post "https://lake.ayon.cdloud/api/v1/statistics": dial tcp: lookup lake.ayon.cdloud on 8.8.8.8:53: no such host
Get "https://lake.ayon.cdloud/api/v1/repositories?after=&amount=100": dial tcp: lookup lake.ayon.cdloud on 8.8.8.8:53: no such host
Error executing command.

this is the return if its correct. not 100% sure how we best use it. i guess we can check if an md table exists ?
kinda jancky ngl.

➜  ~ lakectl repo list
+-----------------------+--------------------------------+------------------+-------------------------------+
| REPOSITORY            | CREATION DATE                  | DEFAULT REF NAME | STORAGE NAMESPACE             |
+-----------------------+--------------------------------+------------------+-------------------------------+
| ayon-usd              | 2024-07-04 10:20:54 +0200 CEST | main             | local://ayon-usd              |
| lake-fs-cicd-test-prj | 2024-07-26 09:51:49 +0200 CEST | main             | local://lake-fs-cicd-test-prj |
| test                  | 2024-07-03 22:56:56 +0200 CEST | main             | local://test                  |
+-----------------------+--------------------------------+------------------+-------------------------------+

last option would be to use the rest api with python requests. that might be a bit better in terms of error reporting quality.

[BigRoy]

Ah, no - this is perfect.

When the URL is wrong and when it can't connect due to authentication - both those errors seem 'clear' to me. They report what failed in likely the best way lakectl can - so I'd say, if an error occurs from lakectl let's pass that on.

The lakectl repo list result - that would be the report if it succeeds - in that case, we don't need to log anything and can just continue trying to access the right repo as we were, right?

So TL;DR - it's just a matter of passing on the errors from lakectl if errors occur?

[Lypsolon]

Ah, no - this is perfect.

When the URL is wrong and when it can't connect due to authentication - both those errors seem 'clear' to me. They report what failed in likely the best way lakectl can - so I'd say, if an error occurs from lakectl let's pass that on.

The lakectl repo list result - that would be the report if it succeeds - in that case, we don't need to log anything and can just continue trying to access the right repo as we were, right?

So TL;DR - it's just a matter of passing on the errors from lakectl if errors occur?

yes the last one is the success "status" we just need to correctly detect it but that should not be to hard i think.
we could even check if the lake repo in the settings is on the server by checking the storage namespace.
Pandas would be easy but im not sure if we have it available i would reader not add to the standard dependency's.

[BigRoy]

yes the last one is the success "status" we just need to correctly detect it but that should not be to hard i think.
we could even check if the lake repo in the settings is on the server by checking the storage namespace.
Pandas would be easy but im not sure if we have it available i would reader not add to the standard dependency's.

No, don't do that - don't make it bigger than it is. We need to find the hardest 'critical' failure cases which is the user having the wrong server OR the wrong lakeFS API keys (I think the latter will be most common for new users)

[Lypsolon]

[Reference Info]
PR for the repo list function we can use to get connection info.
ynput/ayon-bin-bridge-client#20