Evaluation of User Experience (UX) in online systems is essential but also costly in terms of both time and money. We are interested in building automatic tools to assist development teams in evaluating the UX even with a limited budget and short iteration cycles. Analyzing user interaction may provide a good indication of the effort that users are required to spend on certain page elements, which we consider an important factor of UX. In this paper we present UX-Analyzer, a web tool that automatically calculates and displays the user interaction effort of web pages. The interaction effort is a score that represents the extra burden, difficulty or discomfort of a user when interacting with a specific webpage element. This score has been learned from manual UX expert ratings and different measures of interaction logs, called micro-measures. UX-Analyzer further aggregates individual element scores from different users to compose a single score for each analyzed web page. The tool can also show the compared scores among alternative versions of a page. Thus, UX-Analyzer allows UX experts or other team members to evaluate web pages automatically in a transparent way, since the interaction effort may be calculated with multiple users in their real context of use. Visualizing the interaction effort may provide a good indication towards the level of UX of an online system. It may also be used in the context of an A/B testing approach that instead of revenue or conversions, it compares the UX of alternative versions of a website. That is, this approach enables web analytics based on UX, i.e., measuring, analyzing and improving UX quality in each development cycle.