topic Re: Better error handiling in Morpheus cypher package in HPE Morpheus Enterprise Software

Better error handiling in Morpheus cypher package

rajmartha — Mon, 07 Oct 2024 17:34:09 GMT

HI All

i belive morpheus cypher package needs to have a better error handling,.

we are running the below error almost every day when 5 to 8 jobs run at the same time, which use the morpheus cypher package. the reason might be most of our jobs, i guess 5 to 8 of them are scheduled at the same time, out of 8 jobs one the job is erroring out with below error.

Traceback (most recent call last):
  File "<stdin>", line 44, in <module>
  File "<stdin>", line 25, in main
  File "/var/opt/morpheus/morpheus-local/workspace/py-f85459cd-28c3-4ae8-9dc0-19b435a3b701/lib/python3.11/site-packages/morpheuscypher/__init__.py", line 85, in __init__
    self.set_cypher_endpoint()
  File "/var/opt/morpheus/morpheus-local/workspace/py-f85459cd-28c3-4ae8-9dc0-19b435a3b701/lib/python3.11/site-packages/morpheuscypher/__init__.py", line 103, in set_cypher_endpoint
    if LooseVersion(data['buildVersion']) < LooseVersion('5.3.3'):
                    ~~~~^^^^^^^^^^^^^^^^
KeyError: 'buildVersion'

i downloaded the source code for morpheus cypher package

at line number 103 in def set_cypher_endpoint(self) function,

current code,

    def set_cypher_endpoint(self):
        appliance_url = self.url
        url = appliance_url + "/api/ping"
        headers = {'content-type': 'application/json', 'Authorization': "BEARER %s" % self.token}
        with warnings.catch_warnings():
            warnings.filterwarnings("ignore", category=Warning)
            r = requests.get(url=url, headers=headers, verify=self.ssl_verify)
        data = r.json()
        ***if LooseVersion(data['buildVersion']) < LooseVersion('5.3.3'):***
            self.cypher_endpoint = "/api/cypher/v1/"
        else:
            self.cypher_endpoint = "/api/cypher/"

i think the code should be

def set_cypher_endpoint(self):
    appliance_url = self.url
    url = appliance_url + "/api/ping"
    headers = {'content-type': 'application/json', 'Authorization': "BEARER %s" % self.token}
    
    with warnings.catch_warnings():
        warnings.filterwarnings("ignore", category=Warning)
        r = requests.get(url=url, headers=headers, verify=self.ssl_verify)

    try:
        data = r.json()
        # Check if 'buildVersion' is present
        if 'buildVersion' in data and LooseVersion(data['buildVersion']) >= LooseVersion('5.3.3'):
            self.cypher_endpoint = "/api/cypher/"
        else:
            # Use the fallback endpoint if buildVersion is missing or < 5.3.3
            self.cypher_endpoint = "/api/cypher/v1/"
    except (ValueError, KeyError):
        # Fallback to /api/cypher/v1/ if there's a JSON error or missing 'buildVersion'
        self.cypher_endpoint = "/api/cypher/v1/"

who ever was developing that package, can you please take a look?

Re: Better error handiling in Morpheus cypher package

rajmartha — Wed, 16 Oct 2024 13:34:38 GMT

@cbunge any suggestions on how we can get the apitoken from the below code?

self.token = morpheus['morpheus']['apiAccessToken']

Re: Better error handiling in Morpheus cypher package

cbunge — Wed, 16 Oct 2024 14:14:33 GMT

I believe the var morpheus.executionLeaseToken allows concurrency for cypher access

Re: Better error handiling in Morpheus cypher package

cbunge — Wed, 16 Oct 2024 14:11:47 GMT

That’s a run once token during an execution and immediately expires the next time that variable is called or when the execution completes.

Re: Better error handiling in Morpheus cypher package

rajmartha — Tue, 08 Oct 2024 16:37:58 GMT

support ticket i opened for this issue has been closed,since this needs to be addressed here. can someone from Morpheus fix this issue please?

Re: Better error handiling in Morpheus cypher package

rajmartha — Tue, 22 Oct 2024 16:14:22 GMT

@cbunge any update on this?

Re: Better error handiling in Morpheus cypher package

rajmartha — Tue, 22 Oct 2024 16:19:39 GMT

@ncelebic @cbunge we are keep getting this exceptions, can one of you please help us here?

Traceback (most recent call last): File "<stdin>", line 44, in <module> File "<stdin>", line 25, in main File "/var/opt/morpheus/morpheus-local/workspace/py-605bc1f9-1e72-4027-9244-d75767499d25/lib/python3.11/site-packages/morpheuscypher/__init__.py", line 122, in get raise Exception(data['msg']) Exception: Unauthorized. You must be authenticated to access this api

Traceback (most recent call last): File "<stdin>", line 44, in <module> File "<stdin>", line 25, in main File "/var/opt/morpheus/morpheus-local/workspace/py-a2c37c59-4866-48ff-80c7-31270aeced57/lib/python3.11/site-packages/morpheuscypher/__init__.py", line 122, in get raise Exception(data['msg']) Exception: Invalid Vault Token

Re: Better error handiling in Morpheus cypher package

rajmartha — Thu, 24 Oct 2024 13:55:14 GMT

@Tyler_Boyd did not see the error so far,.

Re: Better error handiling in Morpheus cypher package

rajmartha — Mon, 14 Oct 2024 18:47:35 GMT

@ncelebic now i am seeing this error

Traceback (most recent call last):
  File "<stdin>", line 44, in <module>
  File "<stdin>", line 25, in main
  File "/var/opt/morpheus/morpheus-local/workspace/py-772fabfd-e479-46de-9432-6b28a09530d0/lib/python3.11/site-packages/morpheuscypher/__init__.py", line 122, in get
    raise Exception(data['msg'])
Exception: Invalid Vault Token

Re: Better error handiling in Morpheus cypher package

Tue, 08 Oct 2024 17:54:16 GMT

Hi @rmartha,

This module is not a core Morpheus product, and that’s why support won’t handle it directly. The repository is located at GitHub - tryfan/python-morpheus-cypher . I’d be happy to review a pull request or address an issue. Please note that there is no SLA for fixing this as this falls under our policy located here: Morpheus Support

Re: Better error handiling in Morpheus cypher package

rajmartha — Fri, 18 Oct 2024 15:16:04 GMT

@cbunge can you explan more about it? Do you have any sample code on how we cna use the var morpheus.executionLeaseToken ?

Re: Better error handiling in Morpheus cypher package

rajmartha — Tue, 08 Oct 2024 18:52:00 GMT

Hi @ncelebic

here is the pull request

Re: Better error handiling in Morpheus cypher package

rajmartha — Tue, 15 Oct 2024 14:05:38 GMT

@ncelebic we see this error this morning too,

Traceback (most recent call last):
  File "", line 44, in 
  File "", line 25, in main
  File "/var/opt/morpheus/morpheus-local/workspace/py-04d507ad-579b-4f9d-9f14-f1a0e7c5e196/lib/python3.11/site-packages/morpheuscypher/__init__.py", line 122, in get
    raise Exception(data['msg'])
Exception: Unauthorized. You must be authenticated to access this api

While you might not notice the error with a single execution, our scenario involves jobs that repeatedly execute this module more than ten times simultaneously.

Re: Better error handiling in Morpheus cypher package

rajmartha — Mon, 07 Oct 2024 17:58:58 GMT

Tagging @ncelebic @cbunge

Re: Better error handiling in Morpheus cypher package

rajmartha — Wed, 23 Oct 2024 13:19:18 GMT

Thank you somuch @Tyler_Boyd i did see the Pull request, will test and let you know once the code is merged.

Re: Better error handiling in Morpheus cypher package

rajmartha — Wed, 23 Oct 2024 19:58:56 GMT

Thank you @Tyler_Boyd will let you know if i run into any issues

Re: Better error handiling in Morpheus cypher package

tyboyd — Wed, 23 Oct 2024 12:26:50 GMT

You would just use morpheus[“executionLeaseToken”] instead of morpheus[‘morpheus’][‘apiAccessToken’]

executionLeaseToken is valid during the task execution while apiAccessToken is valid until it is called again.

If you are using the cypher module you would use

from morpheuscypher import Cypher

value = Cypher(morpheus=morpheus,ssl_verify=False,token=morpheus["executionLeaseToken"]).get('secret/test')

print(value)

@ncelebic It might be good to update the code to prefer executionLeaseToken over apiAccessToken. Older versions won’t have executionLeaseToken so I could have it fall back to apiAccessToken. I will make a pull request.

Re: Better error handiling in Morpheus cypher package

tyboyd — Wed, 23 Oct 2024 18:59:46 GMT

You should be good to test now.

If you want to specify the version you can set: ‘morpheus-cypher==0.3.1’ in the additional packages field

Re: Better error handiling in Morpheus cypher package

Wed, 09 Oct 2024 13:43:58 GMT

Thanks for the PR! After looking at this entire block more closely, it was causing more trouble than it was worth in testing. This was based on an API path change in version 5.3.3 of Morpheus, which is no longer relevant. I left an escape hatch open in the README, but otherwise just ripped it out.

I just pushed version 0.3.0 to PyPI. Let me know if you have any issues either in this thread, or you can raise an issue in the github repo.

Thanks!

Re: Better error handiling in Morpheus cypher package

Mon, 14 Oct 2024 19:05:11 GMT

I am not getting this error when I run the module. This looks like a token issue in Morpheus as opposed to a module issue. I would need more detail to offer any more insight.