Make sure the Google Calendar for Confluence user has access to the pages that are using calendars. For example, if you want to embed a calendar in a Space that has restricted access, make sure to add the Google Calendar for Confluence user to the list of users that have View permissions:

Google Calendar space permissions

Another issue can be the permissions set on an individual blog post, page (or parent page), where you would need to grant the Google Calendar for Confluence user View permissions as well:

Edit Confluence resttrictions