By default, Nokia N95 is set to use 100mW worth of power which can drain your battery drastically if you frequently use WLAN. You can fix this by going into Tools—>Settings—> connection—>Wireless Lan—>Options—>Advanced settings.You will get a warning which can be safely ignored. Scroll down to Power and change to 10mW which is sufficient for most situations.

You can also turn off Auto scanning for WiFI WLans from the same location as above. Just set Scan for Networks to Never.

Thanks to seki for the tip!