Part of it is money--it gives a person the time to train (rather than working all the time) and access to good equipment and good coaching. Some olympic sports are sports for the rich, but in the US and parts of Europe middle class is rich enough. People in richer countries can send their children to be professionally coached, while in other countries people are sending their children to work. While some countries try to develop programs to identify and develop talent, take a look at gymnastics, figure skating, equestrian, and other expensive individual sports. In the US a lot of parents are able to pay to get their kids training--so even if they didn't show interest or talent in the sport when they were seven years old, they can still get to the top with hard work. Additional leisure time plus access to coaching gives the US an edge over countries like China--who try to find talent young and train it, with no perfect way of knowing in advance whether the child has the work ethic or genetics to grow into a great athlete. Then if you look at team sports--most of the ones that are not equipment-intensive are available at such a low cost that even some of the poorest kids in America can afford to play--and again there is a coaching advantage. (Also, the fall of the soviet union gave many great eastern european coaches and athletes a chance to leave for greener pastures, and many ended up in the US.)
When you look at sports like distance running, the American advantage goes away--in sports where coaching is less important and equipment and money is unimportant, there is very little advantage.
There's a lot of reasons why countries do well or poorly in the medal count at the Olympics. Certainly population is one. And you're right that if medal are distributed proportionately according to population, China's 1.2B people should earn 4x as many medals are the USA and its 300M people. To keep things easy, I'll confine my response to China and the US only.
Someone mentioned wealth/income and leisure time, which I agree with. Yes, China's biggest cities are filled with gleaming new buildings and the wealthy zoom about in expensive European cars. But living conditions in much of central and western China resemble those of a third-world nation. Most citizens there are subsistence farmers, growing food just to survive themselves. An athletic prodigy born there can't foolishly waste time playing games -- fields need to be plowed. We're talking about Dust Bowl / Grapes of Wrath poverty. I don't have any numbers, but the number of Chinese who economically can't participate in sport has got to be in the hundreds of millions.
There are other factors that can't be objectively measured. I believe culture and tradition play a large role in Olympic success. For example, the US has a very strong gun culture, as even casual followers of current news know. So it's not surprising the US does well in shooting events. The same goes for basketball. Studies have shown that the most popular people in American high school are the athletes. I don't know if that's the case in China, but if it isn't, that's a huge factor.
Don't forget that three of America's most popular sports, baseball, football, and softball, are not Olympic sports. If they were, the USA would have a permanent place on thosee medal stands. That's because we all grew up with those sports, another example of how society and tradition affect proficiency in a sport.
There's also the availability of exercise facilities. Daddy plays catch with his child and discovers he/she has a rocket arm. That child's talents will probably be honed with baseball or softball and maybe move on to the javelin. But if there was no park to play catch years before, that talent may not have been discovered.
I'm sure I'm only scratching the surface. Still, I hope you found this helpful. I'll leave it to others to list the factors I've whiffed on.
money!!1
I understand that the United States has a large population, but they still manage to beat China, a country with over 4 times the population of the USA. Why does the United States produce such good athletes? This is a serious question.