"""
This script addresses the situation where a server currently uses USE_NUMERIC_ELEMENT_IDS=True and needs to use
USE_NUMERIC_ELEMENT_IDS=False. The script deletes the 'id' property from the devices and updates the properties
(ctrl_id, Instrument_id, elements, motor_role_ids, counter_role_ids, physical_roles) referencing other devices.
It will also fix invalid configufation of pseudoelements. Before version 3.4, pseudoelements had a property called
`elements` with the ids of physical elements as value. This same value was duplicated in their controller inside the 
property motor_role_ids/counter_role_ids. The script will check that both properties have the same value. If they do
not match, the value from motor_role_ids/counter_role_ids will be overwriten with the value from elements property.
Finally, as part of the cleanup phase this script removes 'pseudo_motor_role_ids' and 'pseudo_counter_role_ids'
from the pseudo element Controller devices and 'elements' property from the PseudoMotor and PseudoCounter devices.


The order in which the script modifies the system is as follows:
1. Fix possible incorrect configuration from elements and motor_role_ids/counter_role_ids.
2. Remove pseudo_motor_role_ids and pseudo_counter_role_ids properties.
3. Remove elements property from pseudomotors and pseudocounters.
4. Update ctrl_id, elements and Instrument_id with alias instead of IDs.
5. Update roles (motor_role_ids, counter_role_ids, physical_roles) with alias instead of IDs.
6. Remove motorgroups.
7. Remove IDs from all devices.


The script has 3 modes of execution:
1. fix_pseudos: Runs point 1 from the list above. As it only fixes incorrect configuration, it can be used either in version 3.3.x or in
   > 3.4.x. If this mode finds problems, it must write these changes to the DB for the rest of the script to run properly.
2. cleanup: Runs point 1, 2 and 3 from the list above. It can only be run with version > 3.4, as it removes properties used in previous versions.
   Even if the system is still using IDs, it is recommended to run it to remove old properties no longer used.
3. migrate: Runs all points from the list above. It can only be used with sardana > 3.4 and if the settings USE_NUMERIC_ELEMENT_IDS
   are set to False.

All three modes will need the argument --write to actually write to the DB. It is advisable to first run it without --write.
If the output looks fine to you, you can proceed with the same mode but this time using --write.


For the script to work, all the devices need to have valid values. If one device has been manually updated,
for instance deleting its ID, the script will update neither the device nor the depending on it.
In the same way, the ID value needs to be numeric for the script to work as expected.
The typical errors, which will need to be fixed manually, are:
- no alias defined for an element
- deleted physical element still referenced by a pseudo element controller
If for some reason the script lost connection with the server in the middle of an execution or simply did just not
finish, it can be run again and it will continue the migration

The typical procedure to run the script is:
1) Make a backup before running the script
2) Run python upgrade_ids.py --server=Pool/test1 --mode=fix_pseudos
3) If there are changes and they look good to you, add --write to save them to the DB.
4) Install Sardana 3.4
5) Run:
   - python upgrade_ids.py --server=MacroServer/test1 --mode=cleanup
   - python upgrade_ids.py --server=Pool/test1 --mode=cleanup
6) If both commands in point 5 do not give error, proceed to add --write to save such changes in the DB.
7) If you want to use aliases, set the configuration sardanacustomsettings.USE_NUMERIC_ELEMENT_IDS=False.
8) Run:
   - python upgrade_ids.py --server=MacroServer/test1 --mode=migrate
   - python upgrade_ids.py --server=Pool/test1 --mode=migrate
9) Now that it is migrated, you may want to also enable sardanacustomsettings.USE_PHYSICAL_ROLES_PROPERTY=True
   This way you will be able to use sardana configuration tool, such as dump or load.
10) Restart the Pool and MacroServer.

The command line arguments for the script are:
Usage: upgrade_ids.py [OPTIONS]

Options:
  --server TEXT                   Server name e.g. --sever=Pool/test1
                                  [required]
  --write                         Write changes to DB. Disabled by default
  --mode [fix_pseudos|cleanup|migrate]
                                  Choose in which mode you want the script to
                                  run. fix_pseudos fixes incorrect
                                  configuration, cleanup removes leftover
                                  properties, migrate changes ids for alias.
                                  Read the script docstring for more details.
                                  [required]
  --help                          Show this message and exit.

Besides having a backup of the server, it is still recommended to run the script without the "--write" option
and check the output file to ensure everything is ok.

Since the output is quite large, you may want to redirect it to a file. In Linux systems you can use:
'>' Redirect only standard output, standard errors (logging) will be shown
'2>' Redirect only standard errors (logging), standard output will be shown
'&>' Redirect standard output and standard error (logging) to a file
"""
import logging
import sys
import click
from tango import Database, DevFailed
from pprint import pprint, pformat

def fix_pseudos_incorrect_configuration(device_list_pseudoelements, db):
    ctrl_ids = {}
    problems = []
    for pseudo_element in device_list_pseudoelements:
        pseudo_element_properties = db.get_device_property(pseudo_element, ['elements', 'ctrl_id'])
        if not pseudo_element_properties['elements']:
            continue
        try:
            ctrl_id = pseudo_element_properties['ctrl_id'][0]
        except IndexError:
            print(f"{pseudo_element} has no value for ctrl_id property. Fix it and run the script again.")
            sys.exit()
        elements = pseudo_element_properties['elements']
        if ctrl_id not in ctrl_ids:
            ctrl_ids[ctrl_id] = elements
            continue
        if list(elements) != list(ctrl_ids[ctrl_id]):
            problems.append("The value of elements property in " + pseudo_element + " does not have the same value as other pseudoelements using the same controller."
                            + " It is an invalid configuration, fix it and run it again")
    if problems:
        logging.error(problems)
        sys.exit()

def fix_pseudos(device_list_pseudoelements, db, write_flag, name_map):
    fix_pseudos_incorrect_configuration(device_list_pseudoelements, db)
    changes = {}
    for dev_name in device_list_pseudoelements:
        dev_properties = db.get_device_property(dev_name, ['elements', 'ctrl_id'])
        if dev_properties['elements']:
            try:
                ctrl_id = dev_properties['ctrl_id'][0]
            except IndexError:
                print(f"{dev_name} has no value for ctrl_id property. Fix it and run the script again.")
                sys.exit()
            try:
                # trying to cast element id to int. If it fails, then it is already an alias,
                # otherwise it is still a numerical id.
                int(ctrl_id)
                ctrl_name = name_map[ctrl_id]
            except ValueError:
                ctrl_name = db.get_device_from_alias(ctrl_id)

            # TODO: this could be improved. So far it works under the assumption that a ctrl can only have one property.
            # Thus it only gets the value from the property that exists.
            ctrl_properties = db.get_device_property(ctrl_name, ['motor_role_ids', 'counter_role_ids', 'physical_roles'])
            property_name = next(k for k, v in ctrl_properties.items() if v)
            old_ctrl_properties = ctrl_properties[property_name]
                
            if list(old_ctrl_properties) != list(dev_properties['elements']):
                changes[ctrl_name] = property_name + ": " + str(old_ctrl_properties) + " --> " + str(dev_properties['elements'])
                new_ctrl_properties = {property_name:dev_properties['elements']}
                if write_flag:
                    db.put_device_property(ctrl_name, new_ctrl_properties)
    
    return changes
            

            
    
    return changes

def remove_elements_property(device_list_pseudoelements, db, write_flag):
    deleted = {}
    for dev_name in device_list_pseudoelements:
        dev_properties = db.get_device_property(dev_name, 'elements')
        if dev_properties['elements']:
            if write_flag:
                db.delete_device_property(dev_name, 'elements')
            deleted[dev_name] = dev_properties['elements']

    return deleted

def remove_pseudo_role_ids(device_list_controllers, db, write_flag):
    property_names = ['pseudo_motor_role_ids', 'pseudo_counter_role_ids']
    deleted = {}
    for dev_name in device_list_controllers:
        dev_properties = db.get_device_property(dev_name, property_names)
        has_pseudomotor = False
        has_pseudocounter = False

        if dev_properties['pseudo_motor_role_ids']:
            has_pseudomotor = True

        if dev_properties['pseudo_counter_role_ids']:
            has_pseudocounter = True

        if has_pseudomotor:
            if write_flag:
                db.delete_device_property(dev_name, "pseudo_motor_role_ids")
            deleted[dev_name] = [("pseudo_motor_role_ids: ", dev_properties['pseudo_motor_role_ids'])]
        if has_pseudocounter:
            if write_flag:
                db.delete_device_property(dev_name, "pseudo_counter_role_ids")
            if dev_name in deleted:
                deleted[dev_name].append(("pseudo_counter_role_ids", dev_properties['pseudo_counter_role_ids']))
            else:
                deleted[dev_name] = [("pseudo_counter_role_ids: ", dev_properties['pseudo_counter_role_ids'])]

    return deleted


@click.command()
@click.option('--server', required=True, help="Server name e.g. --sever=Pool/test1")
@click.option('--write', is_flag=True, help="Write changes to DB. Disabled by default")
@click.option('--mode', type=click.Choice(['fix_pseudos', 'cleanup', 'migrate']), required=True,
              help="Choose in which mode you want the script to run. fix_pseudos fixes incorrect configuration, cleanup removes leftover properties, migrate changes ids for alias. Read the script docstring for more details.")

def main(server, write, mode):
    logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s')
    db = Database()
    server_name = server
    write_flag = write
    classes = db.get_device_class_list(server_name)
    if len(classes) < 1:
        logging.warning(f'There are no classes for this server. Ensure the name is correct: {server_name}')
        sys.exit()

    device_names = list(classes.value_string)[::2]
    class_names = list(classes.value_string)[1::2]

    device_list_clean_with_door = [dev for dev, cls in zip(device_names, class_names)
                                   if cls.lower() not in {"dserver", "pool", "macroserver", "motorgroup"}]
    device_list_clean = [dev for dev, cls in zip(device_names, class_names)
                         if cls.lower() not in {"dserver", "pool", "macroserver", "door", "motorgroup"}]

    device_list_controllers = [dev for dev, cls in zip(device_names, class_names) if 'controller' in cls.lower()]
    device_list_motorgroups = [dev for dev, cls in zip(device_names, class_names) if 'motorgroup' in cls.lower()]
    device_list_pseudoelements = [dev for dev, cls in zip(device_names, class_names) if 'pseudocounter' in cls.lower()
                                  or 'pseudomotor' in cls.lower()]

    updates_output_summary = {}
    failed_output_summary = {}



    # 1st part: gather all the info into collections
    new_id_map = {}  # key=old_id, value=new_id
    name_map = {}  # key=old_id, value=name
    dependency_free_list = []  # list old_id's without dependencies
    dependency_ctrl_list = []  # list of pair(old_id, dependency_old_id) with "ctrl_id" property
    dependency_elements_list = []  # list of (old_id, [dependency1_old_id, ...] with "elements" property
    dependency_instrument_list = []  # list of (old_id, [dependency_1_old_id, ...] with "Instrument_id" property

    def store_pool_ids():
        dev_name = [dev for dev, cls in zip(device_names, class_names) if 'pool' in cls.lower()]
        if len(dev_name) != 1:
            # if there is not a pool device, then this is a MacroServer, no need to continue this function
            return None, None, None

        dev_name = dev_name[0]
        dev_properties = db.get_device_property(dev_name, ['InstrumentList'])
        if len(dev_properties["InstrumentList"]) < 1:
            return None, None, None
        dev_properties_changes = {"InstrumentList": [dev_properties['InstrumentList'], " --> "]}
        old_property_value = dev_properties['InstrumentList']
        new_property_value = None
        for i in range(0, len(old_property_value), 3):
            alias = old_property_value[i + 1]
            try:
                old_id = old_property_value[i + 2]
                int(old_id)
                new_id_map[old_id] = alias
            except ValueError:
                # If old_id cannot be cast to int, it means the pool has already been migrated and [i+2] is actually the
                # next NX.
                return None, None, None
            except IndexError:
                # if the position i+2 does not exist, it means there are only 2 values and, therefore, there is no id.
                return None, None, None

            if new_property_value is None:
                new_property_value = old_property_value[i:i + 2]
            else:
                new_property_value.append(old_property_value[i])
                new_property_value.append(old_property_value[i + 1])

        # its properties cannot be updated until all the other devices with "Instruments_id" have been updated,
        # they are saved for later.
        dev_properties = {"InstrumentList": new_property_value}
        dev_properties_changes["InstrumentList"].append(new_property_value)
        return dev_name, dev_properties, dev_properties_changes

    pool_name, pool_properties, pool_properties_changes = store_pool_ids()

    def store_ids_and_dependencies():
        for dev_name in device_list_clean:
            dev_properties = db.get_device_property(dev_name, ['id', 'ctrl_id', 'elements', 'Instrument_id'])
            try:
                dev_id = dev_properties['id'][0]
            except IndexError:
                # This ensures the device has an "id" property. It addresses the issue where a previous execution of the
                # script has been stopped in mid execution. Deleting the "id" property from the devices is the last step
                # of the script. If the current device has no "id", it means that all the dependencies and roles were
                # already updated in the previous execution. Updating them is no longer needed.
                return True

            try:
                dev_alias = db.get_alias_from_device(dev_name)
            except DevFailed as e:
                # addresses the issue where some devices might be lacking an alias
                msg = e.args[0]
                failed_output_summary[dev_id] = str(msg.desc) + " (" + str(msg.reason) + ")"
                continue

            new_id_map[dev_id] = dev_alias
            name_map[dev_id] = dev_name

            hasProperties = False
            property_names = ['ctrl_id', 'elements', 'Instrument_id']
            for property_name in property_names:
                if len(dev_properties[property_name]) > 0:
                    dependency_id = dev_properties[property_name][0]

                    if property_name.lower() == 'elements':
                        any_int = False
                        # checking all the items on elements since sometimes the first value is an url
                        # and yet some following items are numerical id.
                        for item in dev_properties[property_name]:
                            try:
                                int(item)
                                any_int = True
                                break
                            except ValueError:
                                continue
                        if any_int:
                            dependency_elements_list.append((dev_id, dev_properties['elements']))
                            hasProperties = True
                        else:
                            logging.warning(
                                f' The device {dev_name} already has the {property_name} dependency with the alias: {dev_properties[property_name]}. It will not be updated. ')
                    else:
                        try:
                            int(dependency_id)
                            if property_name.lower() == 'ctrl_id':
                                dependency_ctrl_list.append((dev_id, dependency_id))
                            if property_name.lower() == 'instrument_id':
                                dependency_instrument_list.append((dev_id, dependency_id))
                            hasProperties = True
                        except ValueError:
                            # If dependency_id cannot be cast to int, it means the dependency value is already an alias,
                            # updated on a previous execution of the script
                            logging.warning(
                                f' The device {dev_name} already has the {property_name} dependency with the alias: {dev_properties[property_name]}. It will not be updated. ')

            if not hasProperties:
                # If the execution reaches this line it means the device does not have any dependency to update.
                # Append its ID to the dependency_free_list
                dependency_free_list.append(dev_id)
            else:
                updates_output_summary[dev_alias] = []

        # if they have the same length, it means there are no devices with dependencies left to update.
        if len(dependency_free_list) == len(device_list_clean):
            return True
        return False

    dependencies_and_roles_already_updated = store_ids_and_dependencies()
    if failed_output_summary:
        # Should any of the devices be missing the alias, the script ends here
        logging.error(
            "There are devices without an alias. Stopping the script, no changes were made.")
        logging.error("List of devices giving error:")
        logging.error(pformat(failed_output_summary))
        logging.error("Fix them and then run the script again\n")
        sys.exit()

    print("Validating elements from pseudomotors and pseudocounters...")
    # making sure the values from elements property and motor_role_ids/counter_role_ids are in sync
    # If they are not, elments property value overwrites the motor_roleids/counter_role_ids value.
    role_ids_update_summary = fix_pseudos(device_list_pseudoelements, db, write_flag, name_map)
    print("Done\n")

    if mode == "fix_pseudos":
        print("Finished!\n")
        logging.info(f"role_ids properties updated:\n{pformat(role_ids_update_summary)}\n")
        print("%d updated role_ids" % len(role_ids_update_summary))
        if write_flag:
            print("All changes have been written.\n")
            print("Run the script again with mode=--cleanup to perform a cleanup or --mode=migrate to perform a cleanup and migrate.")
        else:
            print(f"\033[93mNo changes were written. \nUse --write argument.\033[0m")
        sys.exit()

    # simply run a cleanup and finish the script (used with --cleanup when launching the script)
    if mode == "cleanup":
        print("Removing pseudo_motor_role_ids and pseudo_counter_role_ids properties from devices...")
        role_ids_delete_output_summary = remove_pseudo_role_ids(device_list_controllers, db, write_flag)
        print("Done\n")

        print("Removing elements property from pseudocounters and pseudomotors...")
        pseudoelements_output_summary = remove_elements_property(device_list_pseudoelements, db, write_flag)
        print("Done\n")

        print("Finished!\n")
        logging.info(f"Fixed elements properties:\n{pformat(role_ids_update_summary)}\n")
        logging.info(f"Deleted role_ids properties:\n{pformat(role_ids_delete_output_summary)}\n")
        logging.info(f"Deleted elements properties:\n{pformat(pseudoelements_output_summary)}\n")
        print("\n%d fixed elements properties" % len(role_ids_update_summary))
        print("%d role_ids properties deleted" % len(role_ids_delete_output_summary))
        print("%d elements properties deleted" % len(pseudoelements_output_summary))
        if write_flag:
            print("\nAll changes have been written.\nYou can now use --mode=migrate if needed.")
        else:
            print(f"\033[93mNo changes were written. Use --write argument.\033[0m")
        sys.exit()
    
    if mode == "migrate" and (not write_flag) and role_ids_update_summary:
        print("Finished!\n")
        logging.info(f"role_ids properties updated:\n{pformat(role_ids_update_summary)}\n")
        print(f"\033[93mNo changes were written. Use --write argument.\033[0m")
        print("You need to write this changes to fix inconsistencies and be able to continue past this point.")  
        print("You can either run it with --mode=fix_pseudos --write (recommended). Or run it with --mode=migrate --write to run everything at once (not recommended)")
        sys.exit()

    # 2nd part: update dependencies with the new id (alias)
    def update_dependencies():

        # used as a flag when encountering errors. It will let the function finish and print all the errors,
        # but no changes will be made and the script will stop at the end of the function.
        disable_updating = False

        # update dependencies for "ctrl_id"
        for dev_old_id, dev_dependency_old_id in dependency_ctrl_list:
            try:
                dev_properties = {"ctrl_id": new_id_map[dev_dependency_old_id]}
                dev_properties_changes = {
                    "ctrl_id": [dev_dependency_old_id, " --> ", new_id_map[dev_dependency_old_id]]}
            except KeyError as e:
                logging.error("Could not update " + name_map[dev_old_id] + " ctrl_id dependency: " + str(e))
                disable_updating = True
            if disable_updating:
                continue
            if write_flag:
                db.put_device_property(name_map[dev_old_id], dev_properties)
            updates_output_summary[new_id_map[dev_old_id]].append(dev_properties_changes)

        # update dependencies for "elements"
        for dev_old_id, dev_dependency_old_id in dependency_elements_list:
            try:
                dev_properties = {}
                dev_properties_changes = {}
                list_elements_old = []
                for count, value in enumerate(dev_dependency_old_id):
                    try:
                        # addresses the issue were some elements values may be an URI and not an id
                        int(value)
                    except ValueError:
                        dev_dependency_old_id[count] = value
                    else:
                        dev_dependency_old_id[count] = new_id_map[value]
                    list_elements_old.append(value)
                dev_dependency_new_ids = dev_dependency_old_id
                dev_properties["elements"] = dev_dependency_new_ids
                dev_properties_changes["elements"] = [list_elements_old, " --> ", str(dev_dependency_new_ids)]
            except KeyError as e:
                # addresses the issue where some devices might be lacking and alias
                logging.error("Could not update " + name_map[dev_old_id] + " elements dependency: " + str(e))
                disable_updating = True
            if disable_updating:
                continue
            if write_flag:
                db.put_device_property(name_map[dev_old_id], dev_properties)
            updates_output_summary[new_id_map[dev_old_id]].append(dev_properties_changes)

        # update dependencies for "Instrument_id"
        for dev_old_id, dev_dependency_old_id in dependency_instrument_list:
            try:
                dev_properties = {"Instrument_id": new_id_map[dev_dependency_old_id]}
                dev_properties_changes = {
                    "Instrument_id": [dev_dependency_old_id, " --> ", new_id_map[dev_dependency_old_id]]}

            except KeyError as e:
                logging.error("Could not update " + name_map[dev_old_id] + " Instrument_id dependency: " + str(e))
                disable_updating = True
            if disable_updating:
                continue
            if write_flag:
                db.put_device_property(name_map[dev_old_id], dev_properties)
            updates_output_summary[new_id_map[dev_old_id]].append(dev_properties_changes)
        if disable_updating:
            sys.exit()

    print("Updating dependencies...")
    if not dependencies_and_roles_already_updated:
        update_dependencies()

    # 3rd part: update roles (motor_role_ids, counter_role_ids, physical_roles)
    # There is no need to check whether motor_role_ids and counter_role_ids will be replaced by physical_roles,
    # it is done automatically by Sardana. The only important thing here is to update the old numeric ID to the new value

    def update_roles():
        # used as a flag when encountering errors. It will let the function finish and print all the errors,
        # but no changes will be made and the script will stop at the end of the function.
        disable_updating = False
        for dev_name in device_list_controllers:
            property_names = ['motor_role_ids', 'counter_role_ids', 'physical_roles']
            dev_properties = db.get_device_property(dev_name, property_names)

            for property_name in property_names:
                if len(dev_properties[property_name]) <= 0:
                    # skip this iteration if the property does not exist (no values)
                    continue

                old_property_value = dev_properties[property_name]
                changed_property_value = [old_property_value, " --> "]

                if property_name == "physical_roles":
                    try:
                        int(old_property_value[1])
                        for i in range(0, len(old_property_value), 2):
                            old_property_value[i + 1] = new_id_map[old_property_value[i + 1]]
                        new_property_value = old_property_value
                        changed_property_value.append(new_property_value)
                    except ValueError:
                        # If old_property_value[1] cannot be cast to int, it means the dependency value is already an alias,
                        # updated on a previous execution of the script
                        logging.warning(
                            f"The device {dev_name} already has the {property_name} dependency with the alias: {old_property_value} . It will not be updated.")
                        continue
                    except KeyError as e:
                        logging.error("Could not update " + dev_name + " " + property_name + " dependency: " + str(e))
                        disable_updating = True

                else:
                    try:
                        int(old_property_value[0])
                        new_property_value = None
                        for i in range(0, len(old_property_value)):
                            int(old_property_value[0])
                            if new_property_value is None:
                                new_property_value = old_property_value[i:i + 1]
                                new_property_value[i] = new_id_map[old_property_value[i]]
                            else:
                                new_property_value.append(new_id_map[old_property_value[i]])
                        changed_property_value.append(new_property_value)
                    except ValueError:
                        # If old_property_value[0] cannot be cast to int, it means the dependency value is already an alias,
                        # updated on a previous execution of the script
                        logging.warning(
                            f"The device {dev_name} already has the {property_name} dependency with the alias: {old_property_value} . It will not be updated.")
                        continue
                    except KeyError as e:
                        logging.error("Could not update " + dev_name + " " + property_name + " dependency: " + str(e))
                        disable_updating = True
                if disable_updating:
                    continue

                dev_property = {property_name: new_property_value}
                if write_flag:
                    db.put_device_property(dev_name, dev_property)
                dev_alias = db.get_alias_from_device(dev_name)
                if not dev_alias in updates_output_summary:
                    updates_output_summary[dev_alias] = []
                updates_output_summary[dev_alias].append({property_name: changed_property_value})
        if disable_updating:
            sys.exit()

    if not dependencies_and_roles_already_updated:
        update_roles()
    else:
        logging.info(
            "This server either has no dependencies (ctrl_id, Instrument_id, elements, motor_role_ids, counter_role_ids, "
            "physical_roles) or they were already updated in a previous execution of the script, skipping this part.")
    print("Done\n")

    # 4th part: update Pool

    def update_pool():
        if pool_name is not None and pool_properties is not None:
            if write_flag:
                db.put_device_property(pool_name, pool_properties)
            try:
                dev_alias = db.get_alias_from_device(pool_name)
            except DevFailed:
                dev_alias = pool_name
            updates_output_summary[dev_alias] = [pool_properties_changes]
        else:
            logging.info(
                "This server either has no Pool class or it was already updated in a previous execution of the script, skipping this part.")

    print("Updating pool...")
    update_pool()
    print("Done\n")

    # 5th part: remove properties no longer used from all elements and remove motorgroups

    def remove_motorgroups():
        deleted = []
        everything_deleted = True
        for dev_name in device_list_motorgroups:
            try:
                if write_flag:
                    db.delete_device(dev_name)
                deleted.append(dev_name)
            except DevFailed:
                logging.error("Could not delete motorgroup: " + dev_name)
                everything_deleted = False
        if everything_deleted:
            return deleted
        else:
            sys.exit()

    print("Removing motorgroups...")
    motorgroup_delete_output_summary = remove_motorgroups()
    print("Done\n")

    def remove_id_property():
        deleted = {}
        for dev_name in device_list_clean_with_door:
            dev_properties = db.get_device_property(dev_name, ['id'])
            try:
                dev_id = dev_properties['id'][0]
            except IndexError:
                continue
            if write_flag:
                db.delete_device_property(dev_name, "id")
            deleted[dev_name] = "id"
        return deleted

    print("Removing ID property from devices...")
    id_delete_output_summary = remove_id_property()
    print("Done\n")

    print("Removing pseudo_motor_role_ids and pseudo_counter_role_ids properties from devices...")
    role_ids_delete_output_summary = remove_pseudo_role_ids(device_list_controllers, db, write_flag)
    print("Done\n")

    print("Removing elements property from pseudocounters and pseudomotors...")
    pseudoelements_output_summary = remove_elements_property(device_list_pseudoelements, db, write_flag)
    print("Done\n")


    def show_results():
        print("Finished!\n")
        logging.info(f"Fixed elements properties:\n{pformat(role_ids_update_summary)}\n")
        logging.info(f"Changed properties:\n{pformat(updates_output_summary)}\n")
        logging.info(f"Deleted id properties:\n{pformat(id_delete_output_summary)}\n")
        logging.info(f"Deleted role_ids properties:\n{pformat(role_ids_delete_output_summary)}\n")
        logging.info(f"Deleted elements properties:\n{pformat(pseudoelements_output_summary)}\n")
        logging.info(f"Removed motorgroups:\n{pformat(motorgroup_delete_output_summary)}")
        print("\n%d fixed elements properties" % len(role_ids_update_summary))
        print("%d properties updated" % len(updates_output_summary))
        print("%d id properties deleted" % len(id_delete_output_summary))
        print("%d role_ids properties deleted" % len(role_ids_delete_output_summary))
        print("%d elements properties deleted" % len(pseudoelements_output_summary))
        print("%d motorgroups deleted\n" % len(motorgroup_delete_output_summary))

        if write_flag:
            print("All changes have been written.\n")
        else:
            print(f"\033[93mNo changes were written. Use --write argument.\033[0m")

    show_results()


if __name__ == "__main__":
    main()
