At times potato plants will produce a fruit on the above ground plant. These 'fruits' are similar to tomatoes in appearance which is not surprising as Potato and Tomato plants are close relatives. More commonly the plant produces only a blossom which although it is white or lavender otherwise resembles a tomato blossom. The blossom however does not always produce a fruit. The fruits are sometimes solitary and sometimes in clusters.
Not to be confused with TomTatoes which are a grafted tomato and potato plant that produce edible tomatoes above ground and potatoes below. These little potato fruits are not Tomatoes and they are poisonous. They contain excessive amounts of solanine which probably won't kill you, but just make you feel like your dying.
These potato fruits are how potatoes ancestors reproduced, in modern potato cultivation the tuber or potato itself is used to produce new plants. The potato fruits are most common following unusually cool wet summers. Cooler wet summers facilitate the blossoms survival and maturation into potato fruits.
Solanine, in abundance is also found in potato tubers that have been exposed to sunlight while growing, which is why you should never eat a potato that has been exposed prior to harvest for any length of time. The tell tale sign is a greenish tinge of the potato itself.
Saving the potato seeds from these fruits is not advisable unless you are delving into plant breeding. The seeds, if planted will not produce a plant true to the parent variety, or in all likelihood anything remotely near the quality you attain from seed potatoes [tubers].