In my frame of reference I am the observer and not what is being observed. One would argue that you can be both, but even when I am the object under investigation, I still guard the property of the observer, as the sole thing that can interpret and deduce. This leads to an impossible contradiction when you are being investigated by others. You can no longer assume similarity between the two point of views. I like to think of it as seeing a train pass by while walking on the street with a friend, where the assumption that your systems of references are similar is a very good approximation. But what if I am on the train and you are on the ground? What if the train is accelerating? This is what it means to be the object under question, in other people's perception. I am an observer on the train, with no information about how fast its moving and no clear reference point, and I am desperately trying to find the transformation matrix that will get me to the inertial frame - to the objective frame of reference.
The observer on the train knows that their system is not inertial and therefore can judge that when they see the trees moving towards them, it is not the trees that are moving but they. However, what if there was an unknown object, that its properties were undetermined as far as movement is concerned. The observer has no way of knowing if its really stationary or not, and should probably trust more the opinion of the friend that is walking on the street. This gives a certain sense of absurdity to the dominant advice and countless repetitions of the phrase "What is important is what you think of yourself, and not how other people view you". Under which frame of reference? I find this ideology self-blinding; willingly plunging oneself into subjectivity with no regard or desire to find the truth. No one true objective frame exists- that is true- however I am not going to define the motion of my satellite in the frame of the satellite itself, since that would give me no information. The transformation matters. In that sense, yes, we can judge ourselves only by our system of reference, as long as we know how to transform back to a widely accepted one - maybe one defined by a collection of a lot of distant stars for the purpose of not getting affected by their individual motion. In other words, in human relationships the "objective" system is better defined by an ensemble of people, rather than a localized system with no available rotation matrix.
This leads me to think that the only judge that matters is not oneself - quite the opposite - it is other people. Thinking otherwise is an attempt to caress to the hyper-individualism that seems to invade increasingly every side of life and much more every side of the internet. But humans are not an individualistic species. We are wired to care about other people's perception of us, since the opposite risks ostracism from the collective, otherwise interpreted by the human brain as a quick and easy path towards death. Forcing ourselves to dismiss our need for external approval and assessment is a voluntary act of emotional self-blindness.