Wordpress is CMS - system for website's content management. It's easy and powerful, and compatible with most of servers, but as a payment, it's also heavyweight and not optimized.
The first reason, why Wordpress website is working slow - is Wordpress. His engine designed for executing step by step, few tens of PHP scripts. You're running index file (index.php), it calls wp-blog-header.php,it calls wp-load.php and template-loader.php, they are calls wp-config.php, functions.php, load.php, version.php, template-loader.php is requesting embed-template.php and wp-trackback.php and so on. And I described only 10% of all running files, without template, without plugins and so on.
Of course, each execution requires a CPU time, a network resource, a PHP attention.
The second reason, why Wordpress is working slow - method of execution. The main issue - PHP/Wordpress will pause content output, until it will not fully downloaded. In a case of HTML files, they will loaded and displayed immediately. If there are something heavyweight, HTML will load it, and also will display previously loaded data. PHP and Wordpress will output only when all amount of info will be loaded.
The third and main reason - not optimized hosting (for Wordpress) . As I told earlier - Wordpress is using multiple PHP requests, and server must working perfectly to process them all, without freezing or slowness.
The best web-server structure to process this - nginx+php-fpm+mariadb. Those fearsome words just names three minimal requirements for each modern website: directly web-server - application which accept requests from outside and outputs answer; script which executing website content (PHP), and database used by website content. Your browser asking nginx to process index.php, nginx runs php-fpm to process this index.php, index.php might use MySQL to process requests, returns response to php-fpm, php-fpm returns response to nginx, nginx returns response to browser. Something like that.
Why nginx? Because it light, fast, easy in use, configurable and modern. In difference of Apache, which is heavyweight, but supports more features as plugins (the same structure which slowdown Wordpress, yep).
Why php-fpm ? Because it's daemon (application which always running in memory), and this allows our web-server to save time instead of run php from disk for each request.
Why MariaDb ? Because this is transparent fork of MySQL (with 100% compatibility) and works a little bit faster. For a 15%-20%, but if this is transparent upgrade for MySQL (doesn't require source-code changes) - why not ?
Please, always keep in minds: the root of Wordpress'es slowness is - incorrect server configuration. Not plugins, not amount of images, and not a content size. Just server configuration.
So, the first solution to increase speed of Wordpress site - creating of proper server configuration. You must use nginx as light web-server, which very smoothly processes requests to small static files, like images and css. And it can keep them in memory, decreasing requests to your drive and increasing speed of output. Absolutely, you must use php-fpm. Do not listen anyone who recommends to do some proxy using apache. In this case, you will have a slow 3-head monster: nginx, will run apache, apache will run php. Nginx is able to work directly with php.
If you hosted under shared hosting (around with other websites, yes) - migrate to your own server (VPS). In the current century, average price of VPS enough for your website - 4-7 USD per month. Do not try to install optimization plugin over another optimization plugin - your website will not be faster, because the root of issue is in server config, but plugin has no access there.
Do not listen suggestions about "hey man, use CDN". CDN (Content Delivery Network) improving delivery (!!!) speed for generated content. And only for visitors, physically far from server. But slowness is at content generation, not delivery. Let's imagine, content generation is taking 5 seconds. And content size ig 500 Kb. Server is located at USA. Visitor opening your website from London. In this case, after visitor will press enter, your server will process his request, generate content (5 seconds), output it, and Internet will transfer those 500 Kb to visitor in 200 ms (of course if visitor's network speed is the same as server's), so the time between pressing "Enter" by visitor, and finish page loading will be 5.2 seconds. In a case of using CDN, this content will be transferred in 50 ms, but it will be generated in the same 5 seconds. So, the total time for website loading will be 5.05 seconds. Well, 195ms - is large difference, right?
Of course this suggestion is related to one type of slowness - when web-browser freezes for few seconds, and then display content. If you have a lot of content (starting from 3-4 Mb), and you might have visitors from other countries - you can use CDN.
About memcached, varnish, redis, and other "super-boosters". Those applications are based on storing of frequently asked information in memory. But they are useless, if your website engine doesn't manage them. In other words, your website should be designed to work with those caches from scratch, and if you will just install them - you just spent some disk space and other server resources, including a little bit of speed. Wordpress was not designed to work with those types of cache, so you shouldn't use them.
Few words about "WP super mega giga cache" and other. The "cache" assumes only one - storing of request results in temporary memory to prevent frequent processing for the same results. For example, let's imagine, we're using the system "nginx-php-website". Our website is calculator. And we request to calculate some big formula. sin(5+5)*735 - doesn't matter. Website is calculating this formula in 2 seconds. So, slowness is there. Doesn't matter what plugins you're using, and how much of CDN are using. The slowness of website caused by engine. If we will use cache - we will use full website engine to calculate our "sin(5+5)*735" only once. Cache will store result in memory, and next time, when we will request the same formula - our cache will return result directly after request, without using "php-website" area. The plus of using cache is visible - speed. The minus of using cache - features of website which will prevent to use cache for everything. Let's imagine, our visitors are using a million of different formulas. In this case, our server can't store all results due to insufficient memory. Possible results are larger than you can imagine. Only one different symbol in next request - and it should be cached once again. The next minus is relevance of output data. Let's imagine, we have our own cached social network and looking for females 16-21 age there. Our website is getting external request from visitor (usually GET, something like &action=search&age1=16&age2=21), and returns cached result, and cache as you know, based on request. But what will happen, if new girl will registered there ? Your social network will have new user on a fact, but each visitor who will run the same request as I exampled - will have non-actual data, without newly registered girl. So, cache is not universal solution. But if you have static data, which will updated only by you (without visitors' comments and so on), using of cache will be the best solution.
Even in a cases if you want use a cache - there is no reason to install "WP super mega giga cache" and other. They will not increase speed as you expected. Will be better if you will use internal server cache. Fastcgi cache, or at least MySQL query cache.
professional server optimizer.