During the time of year when the water coming into the house is colder than the temperature inside the house, this phenomenon can occur. Cold water holds more oxygen than warm water does, consequently when the cold water from the water mains outside come inside our warm homes, and the water begins to warm, the oxygen has to escape. It does so by bubbling out in air bubbles which makes the water look milky. A visual example of this is to run water into a clear container and observe for a short time. If the water clears from the bottom to the top of the container then the phenomenon described is occurring. The air bubbles are moving from the bottom to the top of the container to escape into the open atmosphere.
Here is a video example.