Joomla makes use of php a lot, so what is php? A very basic, one page, no tricks website is just one html file. A PHP file takes the place of the HTML file and that file is in fact very similar to the HTML file and behaves in a similar manner.
You can also include PHP code inside an HTML file, so PHP is not just a file type but also it is the code language - much like HTML.
The difference between PHP and HTML is just what each can do. PHP is more dynamic compared to static HTML.
The most basic website requires only one file - an HTML file. That file would contain the text shown on that page and all the text would be styled using HTML code. For examle - to set the font, the font size, the color - this would be done within that HTML file.
Websites these days are much more than simple HTML websites, and the first step is to take that styling code and put it into a seperate file. This is called a style sheet, and is a css file.