Joomla makes use of php a lot, so what is php? A very basic, one page, no tricks website is just one html file. A PHP file takes the place of the HTML file and that file is in fact very similar to the HTML file and behaves in a similar manner.
You can also include PHP code inside an HTML file, so PHP is not just a file type but also it is the code language - much like HTML.
The difference between PHP and HTML is just what each can do. PHP is more dynamic compared to static HTML.
Why is PHP Good?
Joomla makes use of PHP to provide a Content Management System (CMS). A CMS is best used when you have a rapidly changing site - where content is being added, edited and updated frequently. PHP allows this.
In an HTML site, any changes you make to the HTML code only affect the page you edit, and you can only change the content by editing the HTML code within the HTML file.
A nice way to explain how PHP works is using a blog as an example. A blog is updated regularly, and that means the front page always displays the latest article to be published.
If you were to do that in HTML, you would have to change the code of the front page to show the new article and you would have to make another page if you wanted to keep the previous article available. With PHP, the article is kept in a database and when you click on a link to the main page, the PHP says "Get the latest article from the database" and it is inserted automatically. The previous article likewise just gets pushed to page 2 or to a page of it's own.
PHP can do a lot of other cool stuff to that makes a site dynamic and much easier to change as you want to.
Examples of PHP on this Site
This whole page, and every page is displayed via PHP. One example of how PHP can be more dynamic than HTML is the "Cool Tricks" link of the top menu. Click that link and the page shown is the latest article I have added, similar to the way a blog works. Not only that, but I have limited it (via the Joomla menu options) to show only the latest article that I have added to the "Cool Coding Tricks" Catagory. Other new articles don't count.
And with Joomla, I didn't have to do anything tricky to make this work. I simply set up the Menu Item as Articles » Category / Blog. Joomla and PHP do the rest.
PHP can do this because opposed to HTML which says "show what I have got in this file (as well as these images and videos I am pointing to)" PHP says something like "show all this stuff I want you to get from the database, (as well as what I have got in this file and these images and video I am pointing to)".