Yes, you can apply antibiotic ointment like Polysporin on your baby's injured finger. However, if the redness and irritation do not improve after a few nights, it would be best to consult a doctor. You can consider using a video consultation service due to the current COVID-19 situation. If your regular doctor does not offer this service, you can use platforms like Hello Health.