Because he fucking meant it.
1. Oliver knows how Felicity feels about him, he would not purposefully play with her feelings in that way, even to defeat Slade.
2. He never denied it at the end, on the beach. He just smiled as she babbled.
3. Oliver Queen is a terrible liar. Do we not remember the lattes and sports bottles?
Felicity kidnapped by The Count: “He had you and he was going to hurt you. There was no choice to make.”
L being kidnapped by Slade’s masked man: “The city comes first.”