I am not aware of any existing skill code from Cadence, but it is not difficult to write. Your idea would work. We did it a different way and maybe you'll find it simpler. When you flatten an instance one level, the lower level instance names are preserved if the name is unique at the current level. Given this, here is how we implemented it:
1. Only apply this functionality when flattening one level (although you could handle multiple levels of flattening, this simplfies your code and will still work if you flatten one level again).
2. Only apply this code if instances in the lower level are named for VXL (the leading pipe on the instance name indicates that it was probably named for VXL). You don't gain anything if your lower level instance names are random. While looking at the instances names in the instance you wish to flatten, create the hierarchical instance name for each named instance and store it in a table. The key to the table is the current instance name of the lower level instance. The value is the hierarchical instance name that is made by pre-pending the current instance's name (the instance to be flattened) to the lower level instance name. myTable["|inst1"] = "|topInst|inst1"
3. Rename all the instances at the top level with a suffix that shouldn't be found on any lower level instance names. This will ensure that lower level instance names will be preserved when the instance is flattened. |topInst becomes |topInst_preflatten
4. Flatten the instance one level.
5. Find all the instances now at the top level whose name exists in the myTable created in step 2. Rename these instances with the hierarchical name. when(myTable[inst~>name] inst~>name = myTable[inst~>name])
6. Rename all instances with the suffix (added in step 3) to not have the suffix anymore.
This is just a few simple foreach loops and works well for us.