Holy Day of Obligation. The history of All Saints’ Day dates back to the fourth century, when various feasts were held in honor of Christian saints and martyrs at different times of the year. There was no unified date for a celebration of the saints until 609 or 610, when Pope Boniface IV consecrated the Pantheon in Rome to the Blessed Virgin Mary and all the martyrs, and ordered an annual day of commemoration. At this time, the event was held on 13th May. However, it began to be celebrated on 1st November after Pope Gregory III gave a famous speech on this date in the eighth century.