Question:
Does the new day actually begin at sunset in the Islamic reckoning of time?

Answer:
Daytime starts, with respect to matters of Islamic Law, at dawn (Fajr time) and ends at sunset (Maghrib time).
Nighttime starts at sunset and ends at dawn.
The new calendar day commences at the start of the night, meaning sunset. This means that if you say that you will do something on the night of the 23rd, this means that you will do so after sunset following the day of the 22nd.
There are a few legal exceptions to this rule regarding certain matters of worship. For instance, the day of Eid during Hajj for the pilgrims is followed by the night during which one throws the stones for those who were not able to throw during the day.
And Allah knows best.

Responded:
Sheikh Fahd B. `Abd Al-Rahm�n Al-Yahy�, professor at Al-Im�m University in Al-Qas�m