Do we need to pay our remote employees for a full day if they lose internet access for a few hours?

It depends on their status under the Fair Labor Standard Act (FLSA). Your exempt employees need to be paid their full salary for the day if they did any work before or after the outage (or both).

Your nonexempt employees only need to be paid for time worked. So, if your employees didn’t do any other work while their internet was down, you don’t have to pay them for those hours. Note that any time they spent troubleshooting the connection, either by themselves or with your IT department, would likely qualify as work and should be paid.