There are many nice places to swim around the area in the summer. On the Li River up stream from Yangshuo there are several nice places to swim. One of them has a beach and if you swim across the river there are rocks you can jump and dive off.
However the Yulong River is even nicer, the water is cleaner as motor boats are not allowed. 
The dots on the map show some of the possible swimming locations both on the Yulong River and on the Li River.
A few useful warnings
Be careful of sharp objects on the bottom of the river. Often there is broken glass and other sharp objects hiding on the bottom.
Try not to take anything of value with you. Bags that have been left unattended have gone missing in the past.
Make shore your swimming ability and physical fitness is suitable to look after your self in the river. Some times there are strong currents in the water and the river is very deep in places.