Los Angeles has hosted the Olympics twice (1932, 1984) and each time Olympic Football (as it's called by the IOC, so we're sticking with it) underwent some sort of change. In 1932 it was the removal of the event; he last time the Olympics haven't featured a football tournament. The idea was to instead hold an exhibition for American Football.
The demonstration (i.e. non-medal sport) event featured seniors from Cal, Stanford, and USC taking on seniors from Harvard, Princeton, and Yale. The World's Fair in Chicago the following year held a similar All Star Game, as the seeds started to be planted for professional American football in the United States.
American style football didn't catch on internationally, and remains a uniquely North American sport (Canada and Mexico both field teams). When Los Angeles received the Olympics again in 1984, it chose baseball and tennis as it's demonstration sports. It went full steam ahead with a football tournament.
Four stadiums across the country would host games: Boston's Harvard Stadium, Navy Stadium in Maryland, Stanford Stadium in Palo Alto and the Rose Bowl. Groups A and B were back east, C and D on the West Coast. All of the knockout stage matches were held at the Rose Bowl.
What made the '84 tournament noteworthy is that it was the first time the IOC allowed professional players to participate. FIFA, not wanting the Olympics to rival the World Cup, limited squads to players with no more than five senior squad caps, regardless of age.
As the host nation, the US automatically qualified, and were put in a group with Italy, Eqypt, and Costa Rica. The squad started out strong, with a 3-0 victory over Costa Rica. They got a brace from Rick Davis, who made 129 appearances with the New York Cosmos and continued playing after the league went indoor. The midfielder was captain of the US team for most of the 80s, is in the National Soccer Hall of Fame,
The third goal was scored by Jean Willrich, German-American who made 172 appearances with the San Diego Sockers. If there's a theme here, it's that professionalism came four years too late for the US, as these NASL players had perhaps played past their prime.
The US was held scoreless in their match against Italy, and then it was up to the final group match against Egypt who also came into the match with two points (it was two instead of three back then). Gregg Thompson, 1983 NASL Rookie of the Year (okay so they weren't all in the twilight of their career) scored the lone goal for the US. The match ended in a 1-1 draw.
The teams were even on goal differential, but Egypt beat Costa Rica 4-1. Perhaps that was the tiebreaker that sent Egypt to the knockout stage over the US, it's not really clear.
Anyhow, France ended up winning the gold medal, with Brazil taking silver. The Eastern bloc countries boycotted the 1984 Olympics, but one of the lasting changes of the rule change was allowing the traditional football powers to field squads able to compete with communist countries where amateur athletics were a full time job. Then the whole Communism falling thing happened.
The five cap rule eventually evolved into the system we have now, where players have to be U-23 outside of three over 23 spots. Those over 23 spots typically are rewards to players at the twilight of their career. David Beckham was famously passed over for a spot on Team Great Britain, with Ryan Giggs getting the captain role.
The United States failed to qualify this year, an event that has been blamed for everything from Brek Shea's attitude to Landon Donovan's ennui. While the senior team is currently a mix of some MLS stars but mostly European players, the Olympic team was heavy on MLS players. What people hoped to see this year, and will hope again to see in Brazil, is MLS' youth getting a chance to show the world what they can do.
The Olympic tournament is not as prestigious as the World Cup. FIFA needn't worry. However, as development of talent becomes more and more valuable, the US U-23 team needs to perform at the level the senior team is expected to.