Thanks to Jaap Scherphuis

To change an individual light, press it and its four neighbours.

To know whether you need to press a button or not, check its own light and the neighbouring buttons. If an odd number of these 5 buttons are red, then the button needs to be pressed, otherwise it does not.

Solution: Use the second fact above on all the buttons in the middle two rows. For each button that is red in row 2, press the button above it in row 1. For each button that is red in row 3, press the button below it in row 4.