Although Harry knew that the word appearing on his right cheek was the first thing he'd ever say to his soulmate and not a descriptor of himself, it didn't stop other people from thinking otherwise. Because of the rumors his relatives spread around and the prominent display of the word on his face, Harry was generally treated like a liar while growing up. This didn't change when he began attending Hogwarts either. He was regarded with suspicion, and people who might have otherwise wished to get close to him didn't.
Being called a liar so much in his short life led Harry to be a very honest person. He didn't want the accusations about him to ever become true. He also very much despised liars. It was clearly because his soulmate would lie to him upon their first encounter that he was in the predicament he was in. He always kept an eye out for words that read like lies on other people's bodies as a result.
Harry was sorted into Hufflepuff. While many people understood that the word on his face was what he would first say to his soulmate, most still subconsciously applied the label to him. Furthermore, many felt that it said something negative about Harry if his soulmate was bad enough for him to have to call them a liar before anything else.
Harry had little luck with making friends although he did manage to get along with a Gryffindor named Hermione Granger who was ostracized by her House and shunted by her soulmate and another Gryffindor named Neville Longbottom who no one wanted to work with because he was so inept with magic.
Harry didn't even think to pursue the sorcerer's stone. Instead, he was kidnapped by Professor Quirrell in the middle of the night and brought before the Mirror of Erised after the man realized he wouldn't be able to get the stone himself. It was after Lord Voldemort had been revealed and had made promises to Harry that were in no way likely to be true that Harry couldn't prevent himself from shouting "Liar!" at him.
Harry didn't even have time to register that Lord Voldemort was apparently his soulmate before he had to burn Quirrellmort, causing Professor Quirrell to disintegrate, Lord Voldemort's spirit to leave the body, and Harry to faint from the pain.