I think that helps me understand some, yes, though I apologize if I have misinterpreted.
Are you therefore trying to place J1 and J2 (the comps on the outside with pads on the bottom) mirrored on the board, so that their soldermask pads are correctly on the bottom side? This would cause their conductor pads to be on the wrong layer, as they get mirrored to the bottom layer from the top. Therefore, removing that extra conductor would make the pads stay on the correct conductor layer?
If that is the case, you have two solutions to get the pads on the correct side. You can either edit the pad definitions for these components, so that their unmirrored pad is on the bottom layer (which will then, placed mirrored, put it on the top layer) or you can mirror these with the edit->mirror command in mirror geometry mode and update the soldermask layer pads in the their padstack definition to be on the bottom soldermask layer.
Should you so desire, you could set the material thickness of the unused dielectric and conductor layer to 0, just so that the overall thickness of the design is more accurate.