Yes, it is possible for small bumps to appear on a baby's cheeks when their teeth are coming in. It is recommended to avoid applying anything on the skin and to avoid direct sunlight. Providing a teething toy and giving them pain relief medication for any discomfort can help.