Yup, I let the mouse go last night. I thought about keeping him for a second but then I realized really didn't want to spend any money on buying even the bare necessities it would take to keep some small animal I found in my backyard. He seemed to be in pretty good health - he was way more alert and energetic than he was the day before - so I let him go in the back yard right around the same place I found him.
I wouldn't be surprised if I see him hanging around though. I ended up staying outside to do some weeding last night and the mouse just hung out around me for the entire time I was out there.