Exception: token not specified in ENV or morpheus 'morpheus' 'apiAccessToken'

[rajmartha]

We have a Pythion script to start the RDS instances through Workflow, Recently we created few jobs to start/stop all the RDS instances using the workflow.

So our Subtenant have 13 RDS instance’s, using the Jobs and a schedule they are stopping the RDS instances everyday evening 8pm ad starting them in the morning 7am,

Since this all 13 Jobs using the same schedule most of the RDS instances are starting/stopping fine, but some of them are getting the below exception, in logs i don’t see any error,

is any one seen this error before?

Task Execution Failed on Attempt 1
Traceback (most recent call last):
File “”, line 29, in
File “/var/opt/morpheus/morpheus-local/workspace/py-55779591-8d13-4453-84ac-3aefb8aa2ab1/lib/python3.7/site-packages/morpheuscypher/init.py”, line 84, in init
raise Exception(“token not specified in ENV or morpheus[‘morpheus’][‘apiAccessToken’]”)
Exception: token not specified in ENV or morpheus[‘morpheus’][‘apiAccessToken’]

[cbunge]

Also, what version of Morpheus are you running into this issue? Did this exist on prior versions?

[KoreyG]

Hi @JacoDeWeerd,

It has been quite a while since this post, so I can’t say I remember all functionality at that time or my experience. That said, I believe you are correct. If you want to see who created the instance, then you can check print(morpheus['instance']['createdByUsername']). If the user has been deleted, then it will probably show null. As you said, print(morpheus[‘username’]) will show the current user’s username.

There used to be functionality that the instance object had its own API key associated it with it. This key was of the user that created it, which would also become null when the user was deleted. Again, it has been a while, but I believe the Cypher Python module at that time referenced the API key that was associated to the instance vs the API key of the currently logged in user morpheus['morpheus']['apiAccessToken'] but that is no longer true (if it ever was lol).

I hope that helps!

[JacoDeWeerd · Accepted solution]

Hi @kgawronski ,

Thank you for the swift response and clearing this out.

Meanwhile my problem is more clear: The morpheus[‘morpheus’][‘apiAccessToken’] was empty. This is related to GUI > user settings > API Access > Morpheus Automation > Regenerate: an error occurred.

Solved with:

• logging in with local admin
• remove my user (that was added through AD identity source)
• log in with my user so a fresh user was created
• user settings > API Access > Morpheus Automation > Regenerate: success!

The apiAccessKey now is present again.

I believe that the removal of an AD group and adding another AD group for my access is the key to these problems. I might log a case for that.

[KoreyG]

You may want to put those commands I mentioned in a separate task and run them, seems the Cypher module is looking for that value and throwing the error before output can happen.

That said, it might help you in a direction but it would be best to place a support ticket for a more thorough troubleshooting.

[cbunge]

Are these all using the same task/workflow and just passing different instance arguments in?

[rajmartha]

We started using this in 5.4.4 and we are in 5.4.5 right now,

i see the errors in both 5.4.4 and 5.4.5

here is the python code i use to create the Client

import boto3

import botocore.exceptions

import botocore.errorfactory

import time

import datetime

import sys

from pprint import pprint

from operator import itemgetter

from morpheuscypher import Cypher

from tabulate import tabulate

import random

from random import randint

from time import sleep

sleeptime = random.randint(5, 15)

#print(“sleeping for:”, sleeptime, “seconds”)

time.sleep(sleeptime)

#print(“sleeping is over”)

grab current time for role and DB Snapshot names

time_stamp = time.strftime("%Y%m%d-%H%M%S")

input_cloud = morpheus[‘customOptions’][‘RDSCLOUDS’]

#input_role_arn = sys.argv[1]

input_role_arn = Cypher(morpheus=morpheus).get((‘secret/AWS’ + str(input_cloud) + ‘role_arn’))

#pprint(input_role_arn)

def assume_role(role_arn, region):

"""Grabs temporary keys of assumed roles from ARN

Args:

    role_arn: ARN of the role you want to assume

    region: AWS Region based on Environment selected input_Environemnt

Returns:

    temp_key: Temporary key for assumed role

    temp_secret: Temporary secret key for assumed role

"""

# set resource and region

client = boto3.client(

    'sts',

    region_name = region

    )

response = client.assume_role(

    RoleArn=role_arn,

    RoleSessionName='RDSTask' + time_stamp

    )

temp_key = response['Credentials']['AccessKeyId']

temp_secret = response['Credentials']['SecretAccessKey']

temp_token = response['Credentials']['SessionToken']

return temp_key, temp_secret, temp_token

def rds_client(temp_key, temp_secret, temp_token, region):

"""Sends Boto3 tasks based on task input

Args:

    temp_key: Temporary key from assume_role function

    temp_secret: Temporary secret ket from assume_role function

    temp_token: Temporary session token from the assume_role function

    task: Input task to run corresponding Boto3 task

    db_name: Name of RDS to run task against

    region: AWS Region based on Environment selected input_Environemnt

Returns:

    AWS JSON message

"""

# set resource and region

client = boto3.client(

    'rds',

    region_name = region,

    aws_access_key_id = temp_key,

    aws_secret_access_key = temp_secret,

    aws_session_token = temp_token

    )

return client

def rds_service(task, db_name, client):

# run task based on task input

rds_instances = []

rdsinstances = client.describe_db_instances()

for i in rdsinstances['DBInstances']:

    rds_instances.append(i['DBInstanceIdentifier'])

if db_name in rds_instances:

    response = "response"

    if task == "start":

        response = client.start_db_instance(

            DBInstanceIdentifier=db_name

        )



    if task == "stop":

        try:

            response = client.stop_db_instance(

                DBInstanceIdentifier=db_name,

                #DBSnapshotIdentifier=db_name+time_stamp

[rajmartha]

Here is the another error i see very as well for the same behavior

Task Execution Failed on Attempt 1
Traceback (most recent call last):
File “”, line 29, in
File “/var/opt/morpheus/morpheus-local/workspace/py-82d0ead8-12ec-44c2-8e01-251e6c0478e5/lib/python3.7/site-packages/morpheuscypher/init.py”, line 85, in init
self.set_cypher_endpoint()
File “/var/opt/morpheus/morpheus-local/workspace/py-82d0ead8-12ec-44c2-8e01-251e6c0478e5/lib/python3.7/site-packages/morpheuscypher/init.py”, line 103, in set_cypher_endpoint
if LooseVersion(data[‘buildVersion’]) < LooseVersion(‘5.3.3’):
KeyError: ‘buildVersion’

[rbmorph]

@ncelebic Any idea why they seem to be hitting the token exception from morpheuscypher on some tasks and not others. Assuming its raising this exception if self.token is None:
raise Exception(“token not specified in ENV or morpheus[‘morpheus’][‘apiAccessToken’]”)

[rajmartha]

@kgawronski i have added those print commands yesterday and i am not able to see the api AccessToken or user name here is the same error i see as well

Task Execution Failed on Attempt 1

Traceback (most recent call last):
File “”, line 29, in
File “/var/opt/morpheus/morpheus-local/workspace/py-c863bc7e-1931-4ed6-bec1-b376d4f9d956/lib/python3.7/site-packages/morpheuscypher/init.py”, line 84, in init
raise Exception(“token not specified in ENV or morpheus[‘morpheus’][‘apiAccessToken’]”)
Exception: token not specified in ENV or morpheus[‘morpheus’][‘apiAccessToken’]

[KoreyG]

I was able to reproduce that error in my environment, hopefully the below will indicate the same. In a different (or the same) python task, place this in your code:
print(morpheus['morpheus']['apiAccessToken'])

Run the task and choose an instance to run it against. That should print out the apiAccessToken for the instance. Run it against a working instance and non-working instance. I assume you’ll receive a “None” back on a non-working system, whereas on a working system you’ll receive an API token back.

If this is the case, this can be due to a user being deleted in Morpheus, which may have been the one that originally deployed the instance(s). These values will be removed upon that user deletion.

This entry should return the username associated with the instance:
print(morpheus['username'])

EDIT 03/14/2025: Since this post some time ago, note the above might be outdated and was mentioned later in this thread. Here are some example variables that might be useful here:

print(morpheus['instance']['createdByUsername']) - User’s username that created the instance. Might be null if the user has been deleted.

print(morpheus['instance']['createdByEmail']) - User’s email that created the instance. Might be null if the user has been deleted.

print(morpheus['instance']['createdById']) - User’s ID that created the instance. Might be null if the user has been deleted.

print(morpheus['morpheus']['apiAccessToken']) - API token of the currently logged in user.

print(morpheus['username']) - Username of the currently logged in user.

print(morpheus['user']['username']) - Another for the username of the currently logged in user.

[rajmartha]

Correct, these all are using the same task/workflow and passing different instance names.

[JacoDeWeerd]

kgawronski:

print(morpheus[‘username’])

Hi all,
I am trying to solve a similar issue and trying to understand what is happening.
Should the code for revealing the user that was deleted be print(morpheus[‘instance’][‘createdByUsername’]) instead of print(morpheus[‘username’]) which only prints the current logged in user?

[KoreyG]

I’ve not tested the scenario myself but if you can reproduce removing and adding AD groups does that, a case may be a good idea. Assuming that there is not an issue there, changing AD groups should not matter, as the user and unique user ID should not change, thus the API key (if already generated) should remain. This might be for another thread or that case

Just to add some additional information that might help with what you found and others that might see this later. You are correct that your API token should be generated prior to using the variable, or at least check it is not null. You can also generate your token via the API and also using some code.

If you use the API method, be sure to use the morph-automation client.

You could also add a task prior to the Python run to generate or return an already generated token. You’ll need to either Shell/Bash or PowerShell for example, which can use Morpheus variables in it and use the following:

echo <%= morpheus.getApiAccessToken() %>

or

# <%= morpheus.getApiAccessToken() %>

One will output it to the output of the task, the other will still generate it but you won’t see anything. Following tasks (not the same one) should be able to use the API token generated from the morpheus[‘morpheus’][‘apiAccessToken’] variable.

[rajmartha]

@kgawronski Thanks for recreating the issue though, but here we are not having issues for the same RDS instances everyday, for example, if an instance start fine today and the same instance will not have the problem tomorrow, basically same RDS instance is not having issues every day.

every day different RDS instances have problem stop/start.

[rajmartha]

@kgawronski I have downloaded the Cypher package from morpheus-cypher · PyPI. When i look at the init.py line number 84,

Some reason the token is getting null when token accessed multiple times at the same time, i think they can put a try block or something to re-try when it is null. I also opened a support ticket with all this information, i hope they will update this package soon,

class Cypher:
def init(self, url=None, token=None, morpheus=None, ssl_verify=True):
self.url = None
self.token = None
self.cypher_endpoint = None
self.ssl_verify = ssl_verify

    if url is None:
        if "morpheus_url" in os.environ:
            self.url = os.environ.get("morpheus_url", None)
        elif morpheus is not None:
            self.url = morpheus['morpheus']['applianceUrl']
    else:
        self.url = url

    if self.url is None:
        raise Exception("url not found or specified in ENV or morpheus['morpheus']['applianceUrl']")
    else:
        self.url = self.url.rstrip('/')

    if token is None:
        if "morpheus_token" in os.environ:
            self.token = os.environ.get("morpheus_token", None)
        elif morpheus is not None:
            self.token = morpheus['morpheus']['apiAccessToken']
    else:
        self.token = token
    if self.token is None:
        raise Exception("token not specified in ENV or morpheus['morpheus']['apiAccessToken']")
    self.set_cypher_endpoint()

[#17]

Ok, starting from the top.

The flow for a token is:

• If you specify token='a87242487...' when using the module, it will use that.
• If you have a morpheus_token environment var set, it will use that.
• If you specify morpheus=morpheus, it will use morpheus['morpheus']['apiAccessToken'] as supplied to the python script from Morpheus.
• Else complain that there’s no token.

The above complaint about the version is that we now require authentication to get the Morpheus version, so the module will fail if the token isn’t accepted by Morpheus in the API query.

Out of curiosity, how did you raise a ticket with the author? I certainly didn’t see anything that I remember, and there’s no github issue.