Plum trees are a fertile fruit which can yield from two to three bushels per year, so it is important to know when to harvest plum trees. The hands down surest way to ensure the time is right for picking plum fruit is by its firmness and flavor. The plums will be becoming soft to the touch and the taste will be sweet and juicy. Hopefully, you have actually eaten a ripe plum at some point in your life and can use this memory as a barometer.