Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/powerbi-refresh.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Power BI scheduled refresh

on:
schedule:
- cron: '0 * * * *' # every hour
workflow_dispatch:

jobs:
refresh:
runs-on: windows-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Run Power BI dataset refresh
env:
POWERBI_TENANT_ID: ${{ secrets.POWERBI_TENANT_ID }}
POWERBI_CLIENT_ID: ${{ secrets.POWERBI_CLIENT_ID }}
POWERBI_CLIENT_SECRET: ${{ secrets.POWERBI_CLIENT_SECRET }}
POWERBI_GROUP_ID: ${{ secrets.POWERBI_GROUP_ID }}
POWERBI_DATASET_ID: ${{ secrets.POWERBI_DATASET_ID }}
POWERBI_NOTIFY_WEBHOOK: ${{ secrets.POWERBI_NOTIFY_WEBHOOK }}
run: pwsh -NoLogo -NoProfile -File scripts\powerbi-refresh.ps1
35 changes: 35 additions & 0 deletions scripts/POWERBI_AUTOMATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
Power BI automation: dataset refresh + monitoring

Overview
- scripts/powerbi-refresh.ps1: Starts a dataset refresh using a service principal (client credentials), polls for completion, and optionally posts JSON notifications to a webhook.
- .github/workflows/powerbi-refresh.yml: example GitHub Actions workflow to run the script on a schedule.

Setup
1. Azure AD app registration
- Create an app registration in Azure AD.
- Under API permissions, add Power BI Service -> Application permission: Dataset.ReadWrite.All (or least required), then grant admin consent.
- Create a client secret and copy ClientId and ClientSecret.

2. Power BI tenant settings
- In the Power BI admin portal, enable service principals and allow the app to access Power BI APIs.
- Optionally add the service principal to the target workspace with appropriate role.

3. GitHub Secrets
- Add these secrets to the repo: POWERBI_TENANT_ID, POWERBI_CLIENT_ID, POWERBI_CLIENT_SECRET, POWERBI_GROUP_ID, POWERBI_DATASET_ID
- Optional: POWERBI_NOTIFY_WEBHOOK (HTTP endpoint to receive JSON notifications)

Usage
- Run locally:
pwsh -File scripts\powerbi-refresh.ps1
(script reads values from environment variables if not passed as parameters)

- Using GitHub Actions:
The provided workflow triggers on schedule and manual dispatch. Update cron and secrets as needed.

Security
- Use a least-privilege service principal and rotate secrets regularly.
- If sending notifications, secure the webhook endpoint.

Notes
- This script uses client credentials. For delegated flows or user-scoped refreshes, adapt the authentication flow.
- Test in a non-production workspace first.
8 changes: 8 additions & 0 deletions scripts/powerbi-refresh.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Example environment variables (do NOT commit real secrets)
# Set these as GitHub repository secrets or environment variables on the runner
POWERBI_TENANT_ID=your-tenant-id
POWERBI_CLIENT_ID=your-client-id
POWERBI_CLIENT_SECRET=your-client-secret
POWERBI_GROUP_ID=your-workspace-id
POWERBI_DATASET_ID=your-dataset-id
POWERBI_NOTIFY_WEBHOOK=https://example.com/webhook
92 changes: 92 additions & 0 deletions scripts/powerbi-refresh.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
param(
[string]$TenantId = $env:POWERBI_TENANT_ID,
[string]$ClientId = $env:POWERBI_CLIENT_ID,
[string]$ClientSecret = $env:POWERBI_CLIENT_SECRET,
[string]$GroupId = $env:POWERBI_GROUP_ID,
[string]$DatasetId = $env:POWERBI_DATASET_ID,
[int]$TimeoutMinutes = 30,
[int]$PollIntervalSeconds = 15,
[string]$NotifyWebhook = $env:POWERBI_NOTIFY_WEBHOOK
)

function Write-Log { param($m) Write-Output "$(Get-Date -Format o) - $m" }

function Get-AuthToken {
param($tenant,$client,$secret)
if (-not ($tenant -and $client -and $secret)) { throw "TenantId, ClientId and ClientSecret are required (env or params)." }
$body = @{ grant_type = 'client_credentials'; client_id = $client; client_secret = $secret; scope = 'https://analysis.windows.net/powerbi/api/.default' }
$resp = Invoke-RestMethod -Method Post -Uri "https://login.microsoftonline.com/$tenant/oauth2/v2.0/token" -Body $body -ContentType 'application/x-www-form-urlencoded'
return $resp.access_token
}

function Start-Refresh {
param($token,$group,$dataset)
$uri = "https://api.powerbi.com/v1.0/myorg/groups/$group/datasets/$dataset/refreshes"
$body = @{ notifyOption = 'NoNotification' } | ConvertTo-Json
try {
Write-Log "POST refresh -> $uri"
Invoke-RestMethod -Method Post -Uri $uri -Headers @{ Authorization = "Bearer $token" } -Body $body -ContentType 'application/json'
Write-Log "Refresh requested successfully."
} catch {
throw "Failed to start refresh: $_"
}
}

function Get-Latest-Refresh {
param($token,$group,$dataset)
$uri = "https://api.powerbi.com/v1.0/myorg/groups/$group/datasets/$dataset/refreshes`?$top=1"
try {
$resp = Invoke-RestMethod -Method Get -Uri $uri -Headers @{ Authorization = "Bearer $token" }
return $resp.value | Select-Object -First 1
} catch {
throw "Failed to get refresh history: $_"
}
}

function Notify-Webhook {
param($webhookUrl,$payload)
if (-not $webhookUrl) { return }
try {
Invoke-RestMethod -Method Post -Uri $webhookUrl -Body ($payload | ConvertTo-Json -Depth 5) -ContentType 'application/json' -ErrorAction Stop
Write-Log "Notification sent to webhook."
} catch {
Write-Log "Failed to send notification: $_"
}
}

# --- main ---
try {
Write-Log "Starting Power BI dataset refresh automation"
$token = Get-AuthToken -tenant $TenantId -client $ClientId -secret $ClientSecret
Start-Refresh -token $token -group $GroupId -dataset $DatasetId

$deadline = (Get-Date).AddMinutes($TimeoutMinutes)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Seconds $PollIntervalSeconds
$latest = Get-Latest-Refresh -token $token -group $GroupId -dataset $DatasetId
if (-not $latest) { Write-Log "No refresh history yet; continuing to poll..."; continue }
$status = $latest.status
$startTime = $latest.startTime
$endTime = $latest.endTime
Write-Log "Latest refresh status: $status (started: $startTime, finished: $endTime)"
if ($status -eq 'Completed') {
Notify-Webhook -webhookUrl $NotifyWebhook -payload @{ status = 'Completed'; groupId = $GroupId; datasetId = $DatasetId; startTime = $startTime; endTime = $endTime }
Write-Log "Refresh completed successfully.";
exit 0
} elseif ($status -eq 'Failed' -or $status -eq 'Unknown' -or $status -eq 'PartiallySucceeded') {
Notify-Webhook -webhookUrl $NotifyWebhook -payload @{ status = $status; groupId = $GroupId; datasetId = $DatasetId; startTime = $startTime; endTime = $endTime; refresh = $latest }
Write-Error "Refresh finished with status: $status"
exit 2
} else {
# InProgress, Queued, etc.
Write-Log "Refresh is $status; waiting..."
}
}
Write-Error "Timeout waiting for refresh to finish after $TimeoutMinutes minutes."
Notify-Webhook -webhookUrl $NotifyWebhook -payload @{ status = 'Timeout'; groupId = $GroupId; datasetId = $DatasetId }
exit 3
} catch {
Write-Error "Unhandled error: $_"
Notify-Webhook -webhookUrl $NotifyWebhook -payload @{ status = 'Error'; message = $_.Exception.Message }
exit 4
}