From 9ce1a067cf6322daf03f142046ac237ddde06cc7 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Mon, 15 Jul 2019 13:58:43 -0700 Subject: [PATCH 01/45] parent ed4cd20d800cbab789b6b516f324498e022a1d02 author Aditya Patwardhan 1563224323 -0700 committer Aditya Patwardhan 1595358509 -0700 parent ed4cd20d800cbab789b6b516f324498e022a1d02 author Aditya Patwardhan 1563224323 -0700 committer Aditya Patwardhan 1595358258 -0700 Add releasePipeline yml --- .../releaseBuild/azureDevOps/releaseBuild.yml | 14 +- .../azureDevOps/releasePipeline.yml | 262 ++++++++++++++++++ .../azureDevOps/templates/nuget.yml | 13 +- .../templates/release-GlobalToolTest.yml | 109 ++++++++ .../templates/release-SDKTests.yml | 98 +++++++ .../release-ValidatePackageNames.yml | 104 +++++++ 6 files changed, 592 insertions(+), 8 deletions(-) create mode 100644 tools/releaseBuild/azureDevOps/releasePipeline.yml create mode 100644 tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml create mode 100644 tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml create mode 100644 tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml diff --git a/tools/releaseBuild/azureDevOps/releaseBuild.yml b/tools/releaseBuild/azureDevOps/releaseBuild.yml index 803dd01c90f..9c509099187 100644 --- a/tools/releaseBuild/azureDevOps/releaseBuild.yml +++ b/tools/releaseBuild/azureDevOps/releaseBuild.yml @@ -119,13 +119,13 @@ jobs: - template: templates/mac-package-signing.yml -- template: templates/compliance.yml - parameters: - parentJobs: - - build_windows_x64 - - build_windows_x86 - - build_windows_fxdependent - - build_windows_fxdependentWinDesktop +#- template: templates/compliance.yml +# parameters: +# parentJobs: +# - build_windows_x64 +# - build_windows_x86 +# - build_windows_fxdependent +# - build_windows_fxdependentWinDesktop - template: templates/nuget.yml parameters: diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml new file mode 100644 index 00000000000..0a8ab078ae3 --- /dev/null +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -0,0 +1,262 @@ +trigger: none + +resources: + pipelines: + - pipeline: releasePipeline + source: Co-ordinated Build Test + trigger: + branches: + - releases/* + - releaseYml + +stages: +- stage: ValidateSDK + dependsOn: [] + jobs: + - job: WinSDK_XXX + pool: + vmImage: windows-latest + steps: + - pwsh: | + Write-Host 'Testing for pre-conditions' + displayName: Test pre-conditions + + - template: templates/release-SDKTests.yml + parameters: + jobName: WinSDK + displayName: Windows SDK Test + imageName: windows-latest + + - template: templates/release-SDKTests.yml + parameters: + jobName: LinuxSDK + displayName: Linux SDK Test + imageName: ubuntu-latest + + - template: templates/release-SDKTests.yml + parameters: + jobName: macOSSDK + displayName: macOS SDK Test + imageName: macOS-latest + +- stage: PRCreation + dependsOn: [] + jobs: + - deployment: CreatePRInMaster + displayName: Update README.md and metadata.json + pool: server + environment: PSReleaseCreatePR + +- stage: ValidateGobalTool + dependsOn: [] + jobs: + - job: WinGblTool_XXX + pool: + vmImage: windows-latest + steps: + - pwsh: | + Write-Host 'Testing for pre-conditions' + displayName: Test pre-conditions + - template: templates/release-GlobalToolTest.yml + parameters: + jobName: WinGblTool + displayName: Global Tool Test Windows + imageName: windows-latest + + - template: templates/release-GlobalToolTest.yml + parameters: + jobName: LinuxWinGblTool + displayName: Global Tool Test Linux + imageName: ubuntu-latest + +- stage: ValidatePkgNames + dependsOn: [] + displayName: Package Names validation + jobs: + - job: ValidatePkgNames + pool: + vmImage: macOS-latest + steps: + - pwsh: | + Write-Host 'Testing for pre-conditions' + displayName: Test pre-conditions + - template: templates/release-ValidatePackageNames.yml + +- stage: StartDocker + dependsOn: [] + displayName: Kick Off Docker Staging build + jobs: + - deployment: PSDockerKickOff + displayName: Start Docker build + pool: server + environment: PSReleaseDockerKickOff + +- stage: ManualValidation + dependsOn: [] + displayName: Manual Validation + jobs: + - deployment: ValidateWinPkg + displayName: Validate Windows Packages + pool: server + environment: PSReleaseWinPkgValidate + + - deployment: ValidateLinuxPkg + displayName: Validate Linux Packages + pool: server + environment: PSReleaseLinuxPkgValidate + +- stage: ReleaseAutomation + dependsOn: [] + jobs: + - job: RAKickOff + steps: + - pwsh: | + Write-Host "Start release automation here" + displayName: Start Release Automation + +- stage: GitHubDraftRelease + dependsOn: + - ValidateSDK + - PRCreation + - ValidateGobalTool + - ValidatePkgNames + - StartDocker + - ManualValidation + - ReleaseAutomation + + jobs: + - deployment: AzureBlobPublic + displayName: Make Azure Blob Public + pool: + vmImage: windows-latest + environment: PSReleaseAzureBlobPublic + strategy: + runOnce: + deploy: + steps: + - download: none + - pwsh: | + Write-Host "Make azure blob public" + displayName: Make Azure Blob public" + + - job: GitHubDraft + displayName: Create GitHub Draft release + pool: + vmImage: windows-latest + dependsOn: AzureBlobPublic + steps: + - pwsh: | + Write-Host "Upload artifacts and create GitHub Draft Release" + displayName: Create GitHub Draft Release + +- stage: GitHubManualTasks + dependsOn: GitHubDraftRelease + jobs: + - deployment: UpdateChangeLog + displayName: Update Change Log + pool : server + environment: PSReleaseUpdateChangeLog + + - deployment: PushTag + displayName: Push Git Tag + pool : server + environment: PSReleasePushTag + + - deployment: MakeDraftPublic + displayName: Make GitHub Draft public + pool : server + environment: PSReleaseDraftPublic + +- stage: PublishPackages + dependsOn: GitHubManualTasks + jobs: + - job: PublishNuget + pool: + vmImage: windows-latest + steps: + - pwsh: | + Write-Host "Publish Nuget packages" + displayName: Publish Nuget packages + + - job: PublishPkgsMsftCom + pool: + vmImage: ubuntu-latest + steps: + - pwsh: | + Write-Host "Publish packages.microsoft.com" + displayName: Publish to packages.microsoft.com + +- stage: ChangesToMaster + dependsOn: PublishPackages + jobs: + - deployment: MergeToMaster + displayName: Make sure changes are in master + pool: server + environment: PSReleaseChangesToMaster + +- stage: ReleaseSnapAndDocker + dependsOn: + - PublishPackages + - ChangesToMaster + jobs: + - deployment: ReleaseDocker + displayName: Release Docker + pool: server + environment: PSReleaseDockerRelease + + - deployment: ReleaseSnap + displayName: Release Snap + pool: server + environment: PSReleaseSnapRelease + +- stage: UpdateDotnetDocker + dependsOn: GitHubManualTasks + displayName: Update DotNet SDK Docker images + jobs: + - deployment: DotnetSDkDocker + displayName: Update .NET SDK docker images + pool: server + environment: PSReleaseUpdateDotnetDocker + +- stage: BuildInfoJson + dependsOn: GitHubManualTasks + displayName: Upload BuildInfoJson + jobs: + - job: UploadJson + pool: + vmImage: windows-latest + steps: + - pwsh: | + Write-Host "Upload build json" + displayName: Upload Build JSON for upgrade notification + +- stage: ReleaseVPack + dependsOn: GitHubManualTasks + displayName: Release VPack + jobs: + - job: VPackRelease + pool: + vmImage: windows-latest + steps: + - pwsh: | + Write-Host "Release VPack" + displayName: Release VPack + +- stage: ReleaseClose + displayName: Finish Release + dependsOn: + - ReleaseVPack + - BuildInfoJson + - UpdateDotnetDocker + - ReleaseSnapAndDocker + - ChangesToMaster + jobs: + - deployment: RetainBuild + displayName: Retain Build + pool: server + environment: PSReleaseRetainBuild + + - deployment: DeleteBranch + displayName: Delete release branch + pool: server + environment: PSReleaseDeleteBranch diff --git a/tools/releaseBuild/azureDevOps/templates/nuget.yml b/tools/releaseBuild/azureDevOps/templates/nuget.yml index 6a0c1ef5334..f07583d2ffd 100644 --- a/tools/releaseBuild/azureDevOps/templates/nuget.yml +++ b/tools/releaseBuild/azureDevOps/templates/nuget.yml @@ -137,10 +137,17 @@ jobs: binVersionOverride: $(SigningVersionOverride) condition: and(succeeded(), eq(variables['SHOULD_SIGN'], 'true')) + - pwsh: | + Copy-Item -Path '$(PackagePath)\*.nupkg' -Destination '$(System.ArtifactsDirectory)\signed\' -Verbose -Force + Copy-Item -Path '$(PackagePath)\globaltool\*.nupkg' -Destination '$(System.ArtifactsDirectory)\signed\' -Verbose -Force + displayName: Fake copy when not signing + condition: eq(variables['SHOULD_SIGN'], 'false') + - powershell: | Import-Module $(Build.SourcesDirectory)\build.psm1 -Force Get-ChildItem -Recurse "$(System.ArtifactsDirectory)\signed\*.nupkg" -Verbose | ForEach-Object { Start-NativeExecution -sb { nuget.exe verify -All $_.FullName } } displayName: Verify all packages are signed + condition: eq(variables['SHOULD_SIGN'], 'true') - task: securedevelopmentteam.vss-secure-development-tools.build-task-antimalware.AntiMalware@3 displayName: 'Run MpCmdRun.exe' @@ -156,7 +163,11 @@ jobs: artifactPath: '$(System.ArtifactsDirectory)\signed' - powershell: | - $null = New-Item -Path "$(System.ArtifactsDirectory)\signed\globaltool" -ItemType Directory + if (-not (Test-Path "$(System.ArtifactsDirectory)\signed")) + { + $null = New-Item -Path "$(System.ArtifactsDirectory)\signed\globaltool" -ItemType Directory + } + Move-Item -Path "$(System.ArtifactsDirectory)\signed\PowerShell.*" -Destination "$(System.ArtifactsDirectory)\signed\globaltool" -Force Get-ChildItem "$(System.ArtifactsDirectory)\signed\globaltool" -Recurse displayName: Move global tool packages to subfolder and capture diff --git a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml new file mode 100644 index 00000000000..2f559f82578 --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml @@ -0,0 +1,109 @@ +parameters: + jobName: "" + displayName: "" + imageName: "" + +jobs: +- job: ${{ parameters.jobName }} + displayName: ${{ parameters.displayName }} + pool: + vmImage: ${{ parameters.imageName }} + steps: + - task: DownloadBuildArtifacts@0 + inputs: + buildType: specific + project: PowerShellCore + pipeline: Co-ordinated Build Test + specificBuildWithTriggering: true + buildVersionToDownload: latest + downloadType: single + artifactName: finalResults + downloadPath: '$(Pipeline.Workspace)' + itemPattern: '*.nupkg' + + - pwsh: | + $dotnetMetadataPath = "$(Build.SourcesDirectory)/source/DotnetRuntimeMetadata.json" + $dotnetMetadataJson = Get-Content $dotnetMetadataPath -Raw | ConvertFrom-Json + + # Channel is like: $Channel = "5.0.1xx-preview2" + $Channel = $dotnetMetadataJson.sdk.channel + + $sdkVersion = (Get-Content "$(Build.SourcesDirectory)/source/global.json" -Raw | ConvertFrom-Json).sdk.version + Import-Module "$(Build.SourcesDirectory)/source/build.psm1" -Force + + Find-Dotnet + + if(-not (Get-PackageSource -Name 'dotnet5' -ErrorAction SilentlyContinue)) + { + $nugetFeed = ([xml](Get-Content $(Build.SourcesDirectory)/source/nuget.config -Raw)).Configuration.packagesources.add | Where-Object { $_.Key -eq 'dotnet5' } | Select-Object -ExpandProperty Value + Register-PackageSource -Name 'dotnet5' -Location $nugetFeed -ProviderName NuGet + Write-Verbose -Message "Register new package source 'dotnet5'" -verbose + } + + ## Install latest version from the channel + + Install-Dotnet -Channel "$Channel" -Version $sdkVersion + + Write-Verbose -Message "Installing .NET SDK completed." -Verbose + + displayName: Install .NET + + - pwsh: | + $branch = '$(Build.SourceBranch)' + $version = $branch -replace '^.*(release[-/])v' + $vstsCommandString = "vso[task.setvariable variable=PowerShellVersion]$version" + Write-Verbose -Message "Version is $version" -Verbose + Write-Host -Object "##$vstsCommandString" + displayName: Set PowerShell Version + + - pwsh: | + $env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 + Import-Module "$(Build.SourcesDirectory)/build.psm1" -Force + Start-PSBootstrap -Force + + $toolPath = New-Item -ItemType Directory "$(System.DefaultWorkingDirectory)/toolPath" | Select-Object -ExpandProperty FullName + + dotnet tool install --add-source $(Pipeline.Workspace) --tool-path $toolPath --version '$(PowerShellVersion)' PowerShell.Windows.x64 + displayName: Install global tool + + - pwsh: | + $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/pwsh.exe" + + if (-not (Test-Path $toolPath)) + { + throw "Tool is not installed at $toolPath" + } + else + { + Write-Verbose -Verbose "Tool found at: $toolPath" + } + displayName: Validate tool is installed + + - pwsh: | + Import-Module "$(Build.SourcesDirectory)/build.psm1" -Force + Start-PSBootstrap -Force + + $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/pwsh.exe" + + $versionFound = & $toolPath -c '$PSVersionTable.PSVersion.ToString()' + + if ( '$(PowerShellVersion)' -ne $versionFound) + { + throw "Expected version of global tool not found. Installed version is $versionFound" + } + else + { + write-verbose -verbose "Found expected version: $versionFound" + } + + $dateYear = & $toolPath -c '(Get-Date).Year' + + if ( $dateYear -ne [DateTime]::Now.Year) + { + throw "Get-Date returned incorrect year: $dateYear" + } + else + { + write-verbose -verbose "Got expected year: $dateYear" + } + displayName: Basic validation diff --git a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml new file mode 100644 index 00000000000..3bcc8e8f4fa --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml @@ -0,0 +1,98 @@ +parameters: + jobName: "" + displayName: "" + imageName: "" + +jobs: +- job: ${{ parameters.jobName }} + displayName: ${{ parameters.displayName }} + pool: + vmImage: ${{ parameters.imageName }} + steps: + - task: DownloadBuildArtifacts@0 + inputs: + buildType: specific + project: PowerShellCore + pipeline: Co-ordinated Build Test + specificBuildWithTriggering: true + buildVersionToDownload: latest + downloadType: single + artifactName: finalResults + downloadPath: '$(Pipeline.Workspace)' + itemPattern: '*.nupkg' + + - task: DownloadBuildArtifacts@0 + inputs: + buildType: specific + project: PowerShellCore + pipeline: Co-ordinated Build Test + specificBuildWithTriggering: true + buildVersionToDownload: latest + downloadType: single + artifactName: metadata + downloadPath: '$(Pipeline.Workspace)' + + - pwsh: | + $dotnetMetadataPath = "$(Build.SourcesDirectory)/DotnetRuntimeMetadata.json" + $dotnetMetadataJson = Get-Content $dotnetMetadataPath -Raw | ConvertFrom-Json + + # Channel is like: $Channel = "5.0.1xx-preview2" + $Channel = $dotnetMetadataJson.sdk.channel + + $sdkVersion = (Get-Content "$(Build.SourcesDirectory)/global.json" -Raw | ConvertFrom-Json).sdk.version + Import-Module "$(Build.SourcesDirectory)/build.psm1" -Force + + Find-Dotnet + + if(-not (Get-PackageSource -Name 'dotnet5' -ErrorAction SilentlyContinue)) + { + $nugetFeed = ([xml](Get-Content $(Build.SourcesDirectory)/nuget.config -Raw)).Configuration.packagesources.add | Where-Object { $_.Key -eq 'dotnet5' } | Select-Object -ExpandProperty Value + Register-PackageSource -Name 'dotnet5' -Location $nugetFeed -ProviderName NuGet + Write-Verbose -Message "Register new package source 'dotnet5'" -verbose + } + + ## Install latest version from the channel + + Install-Dotnet -Channel "$Channel" -Version $sdkVersion + + Write-Verbose -Message "Installing .NET SDK completed." -Verbose + + displayName: Install .NET + + - pwsh: | + $env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 + Import-Module "$(Build.SourcesDirectory)/build.psm1" -Force + Start-PSBootstrap -Force + + $localLocation = "$(Pipeline.Workspace)" + $xmlElement = @" + + + + "@ + + $releaseVersion = Get-Content "$(Pipeline.Workspace)/release.json" | ConvertFrom-Json | Select-Object -ExpandProperty 'ReleaseVersion' + + Set-Location -Path $(Build.SourcesDirectory)/test/hosting + + Get-ChildItem + + ## register the packages download directory in the nuget file + $nugetConfigContent = Get-Content ./NuGet.Config -Raw + $updateNugetContent = $nugetConfigContent.Replace("", $xmlElement) + + $updateNugetContent | Out-File ./NuGet.Config -Encoding ascii + + Get-Content ./NuGet.Config + + dotnet --info + dotnet restore + dotnet test /property:RELEASE_VERSION=$releaseVersion --test-adapter-path:. "--logger:xunit;LogFilePath=$(System.DefaultWorkingDirectory)/test-hosting.xml" + + displayName: Restore and execute tests + + - task: PublishTestResults@2 + displayName: 'Publish Test Results **\test-hosting.xml' + inputs: + testResultsFormat: XUnit + testResultsFiles: '**\test-hosting.xml' diff --git a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml new file mode 100644 index 00000000000..8d3f328439b --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml @@ -0,0 +1,104 @@ +steps: +- pwsh: | + $variable = 'releaseTag' + + $branch = '$(Build.SourceBranch)' + if ($branch -notmatch '^.*(release[-/])') { + throw "Branch name is not in release format: '$branch'" + } + + $releaseTag = $Branch -replace '^.*(release[-/])' + $vstsCommandString = "vso[task.setvariable variable=$Variable]$releaseTag" + + Write-Verbose -Message "setting $Variable to $releaseTag" -Verbose + Write-Host -Object "##$vstsCommandString" + displayName: Set Release Tag + +- pwsh: | + $name = "{0}_{1:x}" -f '$(releaseTag)', (Get-Date).Ticks + Write-Host $name + Write-Host "##vso[release.updatereleasename]$name" + displayName: Set Release Name + +- pwsh: | + $azureVersion = '$(ReleaseTag)'.ToLowerInvariant() -replace '\.', '-' + $vstsCommandString = "vso[task.setvariable variable=AzureVersion]$azureVersion" + Write-Host "sending " + $vstsCommandString + Write-Host "##$vstsCommandString" + displayName: Set Container Name + +- pwsh: | + $azcopy = "C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy\AzCopy.exe" + + & $azcopy /Source:https://$(StorageAccount).blob.core.windows.net/$(AzureVersion) /Dest:$(System.ArtifactsDirectory) /S /SourceKey:$(StorageAccountKey) + displayName: Download Azure Artifacts + +- pwsh: | + Get-ChildItem $(System.ArtifactsDirectory)\* -recurse | Select-Object -ExpandProperty Name + displayName: Capture Artifact Listing + +- pwsh: | + $message = @() + Get-ChildItem $(System.ArtifactsDirectory)\* -recurse -filter *.rpm | ForEach-Object { + if($_.Name -notmatch 'powershell\-(preview-|lts-)?\d\.\d\.\d(_[a-z]*\.\d+)?-1.(rhel|centos).\d+\.x86_64\.rpm') + { + $messageInstance = "$($_.Name) is not a valid package name" + $message += $messageInstance + Write-Warning $messageInstance + } + } + if($message.count -gt 0){throw ($message | out-string)} + displayName: Validate RPM package names + +- pwsh: | + $message = @() + Get-ChildItem $(System.ArtifactsDirectory)\* -recurse -filter *.tar.gz | ForEach-Object { + if($_.Name -notmatch 'powershell-(lts-)?\d\.\d\.\d\-([a-z]*.\d+\-)?(linux|osx|linux-alpine)+\-(x64\-fxdependent|x64|arm32|arm64)\.(tar\.gz)') + { + $messageInstance = "$($_.Name) is not a valid package name" + $message += $messageInstance + Write-Warning $messageInstance + } + } + if($message.count -gt 0){throw ($message | out-string)} + displayName: Validate Tar.Gz Package Names + +- pwsh: | + $message = @() + Get-ChildItem $(System.ArtifactsDirectory)\* -recurse -filter *.pkg | ForEach-Object { + if($_.Name -notmatch 'powershell-(lts-)?\d\.\d\.\d\-([a-z]*.\d+\-)?osx(\.10\.12)?\-x64\.pkg') + { + $messageInstance = "$($_.Name) is not a valid package name" + $message += $messageInstance + Write-Warning $messageInstance + } + } + if($message.count -gt 0){throw ($message | out-string)} + displayName: Validate PKG Package Names + +- pwsh: | + $message = @() + Get-ChildItem $(System.ArtifactsDirectory)\* -recurse -include *.zip, *.msi | ForEach-Object { + if($_.Name -notmatch 'PowerShell-\d\.\d\.\d\-([a-z]*.\d+\-)?win\-(fxdependent|x64|arm32|arm64|x86|fxdependentWinDesktop)\.(msi|zip){1}') + { + $messageInstance = "$($_.Name) is not a valid package name" + $message += $messageInstance + Write-Warning $messageInstance + } + } + + if($message.count -gt 0){throw ($message | out-string)} + displayName: Validate Zip and MSI Package Names + +- pwsh: | + $message = @() + Get-ChildItem $(System.ArtifactsDirectory)\* -recurse -filter *.deb | ForEach-Object { + if($_.Name -notmatch 'powershell(-preview|-lts)?_\d\.\d\.\d([\-~][a-z]*.\d+)?-\d\.(debian|ubuntu){1}\.\d+(\.\d+)?_amd64\.deb') + { + $messageInstance = "$($_.Name) is not a valid package name" + $message += $messageInstance + Write-Warning $messageInstance + } + } + if($message.count -gt 0){throw ($message | out-string)} + displayName: Validate Deb Package Names From a62335c4e98c3aaf8e5d57761dc196b4b1b021f7 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Tue, 21 Jul 2020 13:29:46 -0700 Subject: [PATCH 02/45] Create directory if it does not exist --- tools/releaseBuild/azureDevOps/templates/nuget.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/releaseBuild/azureDevOps/templates/nuget.yml b/tools/releaseBuild/azureDevOps/templates/nuget.yml index f07583d2ffd..5a18354b91e 100644 --- a/tools/releaseBuild/azureDevOps/templates/nuget.yml +++ b/tools/releaseBuild/azureDevOps/templates/nuget.yml @@ -138,6 +138,7 @@ jobs: condition: and(succeeded(), eq(variables['SHOULD_SIGN'], 'true')) - pwsh: | + if (-not (Test-Path '$(System.ArtifactsDirectory)\signed\')) { $null = New-Item -ItemType Directory -Path '$(System.ArtifactsDirectory)\signed\' } Copy-Item -Path '$(PackagePath)\*.nupkg' -Destination '$(System.ArtifactsDirectory)\signed\' -Verbose -Force Copy-Item -Path '$(PackagePath)\globaltool\*.nupkg' -Destination '$(System.ArtifactsDirectory)\signed\' -Verbose -Force displayName: Fake copy when not signing From 840c4d1b3e7c7508de6c732bf2d6e0397abcdc35 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Tue, 21 Jul 2020 15:57:59 -0700 Subject: [PATCH 03/45] Fix SDK tests --- .../azureDevOps/releasePipeline.yml | 17 +---------------- .../azureDevOps/templates/release-SDKTests.yml | 2 +- 2 files changed, 2 insertions(+), 17 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 0a8ab078ae3..a73b914f18a 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -13,14 +13,6 @@ stages: - stage: ValidateSDK dependsOn: [] jobs: - - job: WinSDK_XXX - pool: - vmImage: windows-latest - steps: - - pwsh: | - Write-Host 'Testing for pre-conditions' - displayName: Test pre-conditions - - template: templates/release-SDKTests.yml parameters: jobName: WinSDK @@ -47,16 +39,9 @@ stages: pool: server environment: PSReleaseCreatePR -- stage: ValidateGobalTool +- stage: ValidateGlobalTool dependsOn: [] jobs: - - job: WinGblTool_XXX - pool: - vmImage: windows-latest - steps: - - pwsh: | - Write-Host 'Testing for pre-conditions' - displayName: Test pre-conditions - template: templates/release-GlobalToolTest.yml parameters: jobName: WinGblTool diff --git a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml index 3bcc8e8f4fa..e56e58a3408 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml @@ -71,7 +71,7 @@ jobs: "@ - $releaseVersion = Get-Content "$(Pipeline.Workspace)/release.json" | ConvertFrom-Json | Select-Object -ExpandProperty 'ReleaseVersion' + $releaseVersion = Get-Content "$(Pipeline.Workspace)/metadata/release.json" | ConvertFrom-Json | Select-Object -ExpandProperty 'ReleaseVersion' Set-Location -Path $(Build.SourcesDirectory)/test/hosting From d247fffd3bfc0b9da83fe523f2cc561ebca78d84 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 09:56:42 -0700 Subject: [PATCH 04/45] Make trigger manual for testing --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index a73b914f18a..15e625f23b0 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -2,12 +2,16 @@ trigger: none resources: pipelines: + #- pipeline: releasePipeline + # source: Co-ordinated Build Test + # trigger: + # branches: + # - release/* + # - releaseYml + - pipeline: releasePipeline source: Co-ordinated Build Test - trigger: - branches: - - releases/* - - releaseYml + version: 143428 stages: - stage: ValidateSDK From 9e45b0437a7c820b10a81bfc0c84bbd0d35e7365 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 09:58:50 -0700 Subject: [PATCH 05/45] Fix typo in stage bame --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 15e625f23b0..34ca36bd98d 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -107,7 +107,7 @@ stages: dependsOn: - ValidateSDK - PRCreation - - ValidateGobalTool + - ValidateGlobalTool - ValidatePkgNames - StartDocker - ManualValidation From af96a5b07e6c208a5246eb400eb47d925ef706e9 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 10:01:56 -0700 Subject: [PATCH 06/45] Remove version from pipeline resource --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 34ca36bd98d..ae697cb1f96 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -11,7 +11,6 @@ resources: - pipeline: releasePipeline source: Co-ordinated Build Test - version: 143428 stages: - stage: ValidateSDK From d180688aea0930cab70fad4226342c6be20efa99 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 10:27:41 -0700 Subject: [PATCH 07/45] Add branch in pipeline resource --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index ae697cb1f96..5a5835a41db 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -11,6 +11,7 @@ resources: - pipeline: releasePipeline source: Co-ordinated Build Test + branch: release/v7.1.0-preview.5 stages: - stage: ValidateSDK From 2dc48c89400f45918d5a47a057479e9b95714965 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 10:29:02 -0700 Subject: [PATCH 08/45] Change build name in pipeline resource --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 5a5835a41db..b86b18ac33e 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -10,7 +10,7 @@ resources: # - releaseYml - pipeline: releasePipeline - source: Co-ordinated Build Test + source: Co-ordinated Build branch: release/v7.1.0-preview.5 stages: From cf32324ae1ff5f943c4b58ae185d5d0019477757 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 11:04:08 -0700 Subject: [PATCH 09/45] Add quotes --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index b86b18ac33e..e6e6f935b0f 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -10,8 +10,8 @@ resources: # - releaseYml - pipeline: releasePipeline - source: Co-ordinated Build - branch: release/v7.1.0-preview.5 + source: 'Co-ordinated Build' + branch: 'release/v7.1.0-preview.5' stages: - stage: ValidateSDK From 2c74b49985a15fe801c487fba31ea4ef051c927d Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 11:17:39 -0700 Subject: [PATCH 10/45] Remove branch --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index e6e6f935b0f..f260e0324c3 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -11,7 +11,7 @@ resources: - pipeline: releasePipeline source: 'Co-ordinated Build' - branch: 'release/v7.1.0-preview.5' + #branch: 'release/v7.1.0-preview.5' stages: - stage: ValidateSDK From f7d10cfc2762b4d4e0e2b2393955f9ec6dc8bb3d Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 11:19:01 -0700 Subject: [PATCH 11/45] Fix build name --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index f260e0324c3..1632e5d440b 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -10,8 +10,8 @@ resources: # - releaseYml - pipeline: releasePipeline - source: 'Co-ordinated Build' - #branch: 'release/v7.1.0-preview.5' + source: 'Co-ordinated Packages' + branch: 'release/v7.1.0-preview.5' stages: - stage: ValidateSDK From ac1494c54403fa22b17799a506bc357f858397e0 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 11:20:07 -0700 Subject: [PATCH 12/45] Fix build name --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 1632e5d440b..1eb50dfde14 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -10,7 +10,7 @@ resources: # - releaseYml - pipeline: releasePipeline - source: 'Co-ordinated Packages' + source: 'Coordinated Packages' branch: 'release/v7.1.0-preview.5' stages: From 674bcc5ce898502845bbbbd7f671f8debd63077c Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 12:18:09 -0700 Subject: [PATCH 13/45] Fix sdk download --- .../templates/release-SDKTests.yml | 32 +++++++++++++------ 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml index e56e58a3408..4d3c7c24999 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml @@ -9,17 +9,29 @@ jobs: pool: vmImage: ${{ parameters.imageName }} steps: - - task: DownloadBuildArtifacts@0 + - task: DownloadPipelineArtifact@2 inputs: - buildType: specific - project: PowerShellCore - pipeline: Co-ordinated Build Test - specificBuildWithTriggering: true - buildVersionToDownload: latest - downloadType: single - artifactName: finalResults - downloadPath: '$(Pipeline.Workspace)' - itemPattern: '*.nupkg' + preferTriggeringPipeline: true + artifact: finalResults + patterns: '**/*.nupkg' + + - task: DownloadPipelineArtifact@2 + inputs: + preferTriggeringPipeline: true + artifact: metadata + patterns: '**/*.nupkg' + + #- task: DownloadBuildArtifacts@0 + # inputs: + # buildType: specific + # project: PowerShellCore + # pipeline: Co-ordinated Build Test + # specificBuildWithTriggering: true + # buildVersionToDownload: latest + # downloadType: single + # artifactName: finalResults + # downloadPath: '$(Pipeline.Workspace)' + # itemPattern: '*.nupkg' - task: DownloadBuildArtifacts@0 inputs: From 5b34550e4173e4053d4cb448f9a395a7dccef49b Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 12:45:32 -0700 Subject: [PATCH 14/45] Use pipeline resource --- .../templates/release-GlobalToolTest.yml | 26 +++++++++++-------- .../templates/release-SDKTests.yml | 15 ++++------- .../release-ValidatePackageNames.yml | 2 +- 3 files changed, 21 insertions(+), 22 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml index 2f559f82578..b8a0ae419de 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml @@ -9,17 +9,21 @@ jobs: pool: vmImage: ${{ parameters.imageName }} steps: - - task: DownloadBuildArtifacts@0 - inputs: - buildType: specific - project: PowerShellCore - pipeline: Co-ordinated Build Test - specificBuildWithTriggering: true - buildVersionToDownload: latest - downloadType: single - artifactName: finalResults - downloadPath: '$(Pipeline.Workspace)' - itemPattern: '*.nupkg' + - download: releasePipeline + artifact: finalResults + patterns: '**/*.nupkg' + + #- task: DownloadBuildArtifacts@0 + # inputs: + # buildType: specific + # project: PowerShellCore + # pipeline: Co-ordinated Build Test + # specificBuildWithTriggering: true + # buildVersionToDownload: latest + # downloadType: single + # artifactName: finalResults + # downloadPath: '$(Pipeline.Workspace)' + # itemPattern: '*.nupkg' - pwsh: | $dotnetMetadataPath = "$(Build.SourcesDirectory)/source/DotnetRuntimeMetadata.json" diff --git a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml index 4d3c7c24999..928da03ba90 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml @@ -9,17 +9,12 @@ jobs: pool: vmImage: ${{ parameters.imageName }} steps: - - task: DownloadPipelineArtifact@2 - inputs: - preferTriggeringPipeline: true - artifact: finalResults - patterns: '**/*.nupkg' + - download: releasePipeline + artifact: finalResults + patterns: '**/*.nupkg' - - task: DownloadPipelineArtifact@2 - inputs: - preferTriggeringPipeline: true - artifact: metadata - patterns: '**/*.nupkg' + - download: releasePipeline + artifact: metadata #- task: DownloadBuildArtifacts@0 # inputs: diff --git a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml index 8d3f328439b..79f9085ed68 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml @@ -1,8 +1,8 @@ steps: - pwsh: | $variable = 'releaseTag' + $branch = '$(resource.pipeline.releasePipeline.sourceBranch)' - $branch = '$(Build.SourceBranch)' if ($branch -notmatch '^.*(release[-/])') { throw "Branch name is not in release format: '$branch'" } From d3de2aeba51b6e9c07fdff06a735f3552aba8b09 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 12:52:36 -0700 Subject: [PATCH 15/45] Fixes for consuming pipeline resource --- .../templates/release-SDKTests.yml | 23 ------------------- .../release-ValidatePackageNames.yml | 8 ++++++- 2 files changed, 7 insertions(+), 24 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml index 928da03ba90..79600db98b3 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml @@ -16,29 +16,6 @@ jobs: - download: releasePipeline artifact: metadata - #- task: DownloadBuildArtifacts@0 - # inputs: - # buildType: specific - # project: PowerShellCore - # pipeline: Co-ordinated Build Test - # specificBuildWithTriggering: true - # buildVersionToDownload: latest - # downloadType: single - # artifactName: finalResults - # downloadPath: '$(Pipeline.Workspace)' - # itemPattern: '*.nupkg' - - - task: DownloadBuildArtifacts@0 - inputs: - buildType: specific - project: PowerShellCore - pipeline: Co-ordinated Build Test - specificBuildWithTriggering: true - buildVersionToDownload: latest - downloadType: single - artifactName: metadata - downloadPath: '$(Pipeline.Workspace)' - - pwsh: | $dotnetMetadataPath = "$(Build.SourcesDirectory)/DotnetRuntimeMetadata.json" $dotnetMetadataJson = Get-Content $dotnetMetadataPath -Raw | ConvertFrom-Json diff --git a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml index 79f9085ed68..60ff7989d7b 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml @@ -1,7 +1,13 @@ steps: +- pwsh: | + Get-ChildItem ENV: + displayName: Capture environment + - pwsh: | $variable = 'releaseTag' - $branch = '$(resource.pipeline.releasePipeline.sourceBranch)' + $branch = $ENV:RESOURCE_PIPELINE_RELEASEPIPELINE_SOURCEBRANCH' + + Write-Verbose -Verbose "SourceBranch == $branch" if ($branch -notmatch '^.*(release[-/])') { throw "Branch name is not in release format: '$branch'" From 12de68f87c1c9185b1f2dd7b84dae70ac52866ae Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 13:48:46 -0700 Subject: [PATCH 16/45] Path fixes to use pipeline resource --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 3 --- .../azureDevOps/templates/release-GlobalToolTest.yml | 10 +++++----- .../azureDevOps/templates/release-SDKTests.yml | 4 ++-- .../templates/release-ValidatePackageNames.yml | 2 +- 4 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 1eb50dfde14..96f35567ef6 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -66,9 +66,6 @@ stages: pool: vmImage: macOS-latest steps: - - pwsh: | - Write-Host 'Testing for pre-conditions' - displayName: Test pre-conditions - template: templates/release-ValidatePackageNames.yml - stage: StartDocker diff --git a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml index b8a0ae419de..9537aebb990 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml @@ -26,20 +26,20 @@ jobs: # itemPattern: '*.nupkg' - pwsh: | - $dotnetMetadataPath = "$(Build.SourcesDirectory)/source/DotnetRuntimeMetadata.json" + $dotnetMetadataPath = "$(Build.SourcesDirectory)/DotnetRuntimeMetadata.json" $dotnetMetadataJson = Get-Content $dotnetMetadataPath -Raw | ConvertFrom-Json # Channel is like: $Channel = "5.0.1xx-preview2" $Channel = $dotnetMetadataJson.sdk.channel - $sdkVersion = (Get-Content "$(Build.SourcesDirectory)/source/global.json" -Raw | ConvertFrom-Json).sdk.version - Import-Module "$(Build.SourcesDirectory)/source/build.psm1" -Force + $sdkVersion = (Get-Content "$(Build.SourcesDirectory)/global.json" -Raw | ConvertFrom-Json).sdk.version + Import-Module "$(Build.SourcesDirectory)/build.psm1" -Force Find-Dotnet if(-not (Get-PackageSource -Name 'dotnet5' -ErrorAction SilentlyContinue)) { - $nugetFeed = ([xml](Get-Content $(Build.SourcesDirectory)/source/nuget.config -Raw)).Configuration.packagesources.add | Where-Object { $_.Key -eq 'dotnet5' } | Select-Object -ExpandProperty Value + $nugetFeed = ([xml](Get-Content $(Build.SourcesDirectory)/nuget.config -Raw)).Configuration.packagesources.add | Where-Object { $_.Key -eq 'dotnet5' } | Select-Object -ExpandProperty Value Register-PackageSource -Name 'dotnet5' -Location $nugetFeed -ProviderName NuGet Write-Verbose -Message "Register new package source 'dotnet5'" -verbose } @@ -67,7 +67,7 @@ jobs: $toolPath = New-Item -ItemType Directory "$(System.DefaultWorkingDirectory)/toolPath" | Select-Object -ExpandProperty FullName - dotnet tool install --add-source $(Pipeline.Workspace) --tool-path $toolPath --version '$(PowerShellVersion)' PowerShell.Windows.x64 + dotnet tool install --add-source "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults" --tool-path $toolPath --version '$(PowerShellVersion)' PowerShell.Windows.x64 displayName: Install global tool - pwsh: | diff --git a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml index 79600db98b3..df2ea6085d7 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-SDKTests.yml @@ -48,14 +48,14 @@ jobs: Import-Module "$(Build.SourcesDirectory)/build.psm1" -Force Start-PSBootstrap -Force - $localLocation = "$(Pipeline.Workspace)" + $localLocation = "$(Pipeline.Workspace)/releasePipeline/finalResults" $xmlElement = @" "@ - $releaseVersion = Get-Content "$(Pipeline.Workspace)/metadata/release.json" | ConvertFrom-Json | Select-Object -ExpandProperty 'ReleaseVersion' + $releaseVersion = Get-Content "$(Pipeline.Workspace)/releasePipeline/metadata/release.json" | ConvertFrom-Json | Select-Object -ExpandProperty 'ReleaseVersion' Set-Location -Path $(Build.SourcesDirectory)/test/hosting diff --git a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml index 60ff7989d7b..06977301357 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml @@ -5,7 +5,7 @@ steps: - pwsh: | $variable = 'releaseTag' - $branch = $ENV:RESOURCE_PIPELINE_RELEASEPIPELINE_SOURCEBRANCH' + $branch = $ENV:RESOURCES_PIPELINE_RELEASEPIPELINE_SOURCEBRANCH Write-Verbose -Verbose "SourceBranch == $branch" From 743796b2f79c45c8ef96499929b5128109648cc8 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 14:31:57 -0700 Subject: [PATCH 17/45] Fix global tools and package validation --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 2 +- .../azureDevOps/templates/release-GlobalToolTest.yml | 2 +- .../azureDevOps/templates/release-ValidatePackageNames.yml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 96f35567ef6..c5c313aa4c1 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -64,7 +64,7 @@ stages: jobs: - job: ValidatePkgNames pool: - vmImage: macOS-latest + vmImage: windows-latest steps: - template: templates/release-ValidatePackageNames.yml diff --git a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml index 9537aebb990..a9cd3172127 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml @@ -53,7 +53,7 @@ jobs: displayName: Install .NET - pwsh: | - $branch = '$(Build.SourceBranch)' + $branch = $ENV:RESOURCES_PIPELINE_RELEASEPIPELINE_SOURCEBRANCH $version = $branch -replace '^.*(release[-/])v' $vstsCommandString = "vso[task.setvariable variable=PowerShellVersion]$version" Write-Verbose -Message "Version is $version" -Verbose diff --git a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml index 06977301357..31788609e11 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml @@ -23,7 +23,7 @@ steps: - pwsh: | $name = "{0}_{1:x}" -f '$(releaseTag)', (Get-Date).Ticks Write-Host $name - Write-Host "##vso[release.updatereleasename]$name" + Write-Host "##vso[build.updatebuildnumber]$name" displayName: Set Release Name - pwsh: | From b4c08a07d696c4e5f36daf38888bd7bc64f81ca5 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 15:01:05 -0700 Subject: [PATCH 18/45] Add logging to install global tool --- .../azureDevOps/templates/release-GlobalToolTest.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml index a9cd3172127..6369507d91e 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml @@ -68,6 +68,9 @@ jobs: $toolPath = New-Item -ItemType Directory "$(System.DefaultWorkingDirectory)/toolPath" | Select-Object -ExpandProperty FullName dotnet tool install --add-source "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults" --tool-path $toolPath --version '$(PowerShellVersion)' PowerShell.Windows.x64 + + Get-ChildItem -Path $toolPath + displayName: Install global tool - pwsh: | From e86a2f817ff30c735fbb0e3ee516d09bbc8c2d1b Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 15:07:24 -0700 Subject: [PATCH 19/45] Fix exe name for global tool --- .../azureDevOps/templates/release-GlobalToolTest.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml index 6369507d91e..99bbdb42f93 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml @@ -90,7 +90,9 @@ jobs: Import-Module "$(Build.SourcesDirectory)/build.psm1" -Force Start-PSBootstrap -Force - $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/pwsh.exe" + $exeName = if ($IsWindows) { "pwsh.exe" } else { "pwsh" } + + $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/$exeName" $versionFound = & $toolPath -c '$PSVersionTable.PSVersion.ToString()' From 5f9ded36857b7edfef6de7f732c82691d7eb878c Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 15:38:27 -0700 Subject: [PATCH 20/45] Test end to end --- .../azureDevOps/releasePipeline.yml | 19 +++++++++---------- .../templates/release-GlobalToolTest.yml | 3 ++- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index c5c313aa4c1..44a0f8564c6 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -2,16 +2,16 @@ trigger: none resources: pipelines: - #- pipeline: releasePipeline - # source: Co-ordinated Build Test - # trigger: - # branches: - # - release/* - # - releaseYml - - pipeline: releasePipeline - source: 'Coordinated Packages' - branch: 'release/v7.1.0-preview.5' + source: Co-ordinated Build Test + trigger: + branches: + - release/* + - releaseYml + + #- pipeline: releasePipeline + # source: 'Coordinated Packages' + # branch: 'release/v7.1.0-preview.5' stages: - stage: ValidateSDK @@ -104,7 +104,6 @@ stages: dependsOn: - ValidateSDK - PRCreation - - ValidateGlobalTool - ValidatePkgNames - StartDocker - ManualValidation diff --git a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml index 99bbdb42f93..42dcd460d35 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml @@ -74,7 +74,8 @@ jobs: displayName: Install global tool - pwsh: | - $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/pwsh.exe" + $exeName = if ($IsWindows) { "pwsh.exe" } else { "pwsh" } + $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/$exeName" if (-not (Test-Path $toolPath)) { From cdf55b23d3b4a8af0f221a74b96c91d7ff5000ec Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 15:41:34 -0700 Subject: [PATCH 21/45] Keep using preview5 for test --- .../azureDevOps/releasePipeline.yml | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 44a0f8564c6..a6a5bbd3e92 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -2,16 +2,16 @@ trigger: none resources: pipelines: - - pipeline: releasePipeline - source: Co-ordinated Build Test - trigger: - branches: - - release/* - - releaseYml - #- pipeline: releasePipeline - # source: 'Coordinated Packages' - # branch: 'release/v7.1.0-preview.5' + # source: Co-ordinated Build Test + # trigger: + # branches: + # - release/* + # - releaseYml + + - pipeline: releasePipeline + source: 'Coordinated Packages' + branch: 'release/v7.1.0-preview.5' stages: - stage: ValidateSDK From cff3f8411fe5e89c458cfe68eefdfabee72c1231 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 16:34:37 -0700 Subject: [PATCH 22/45] Fix global tool tests --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 4 ++++ .../azureDevOps/templates/release-GlobalToolTest.yml | 10 ++++++---- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index a6a5bbd3e92..992b1343d9a 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -51,12 +51,16 @@ stages: jobName: WinGblTool displayName: Global Tool Test Windows imageName: windows-latest + globalToolExeName: 'pwsh.exe' + globalToolPackageName: 'PowerShell.Windows.x64' - template: templates/release-GlobalToolTest.yml parameters: jobName: LinuxWinGblTool displayName: Global Tool Test Linux imageName: ubuntu-latest + globalToolExeName: 'pwsh' + globalToolPackageName: 'PowerShell.Linux.x64' - stage: ValidatePkgNames dependsOn: [] diff --git a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml index 42dcd460d35..795bc30b00b 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml @@ -2,6 +2,9 @@ parameters: jobName: "" displayName: "" imageName: "" + globalToolExeName: 'pwsh.exe' + globalToolPackageName: 'PowerShell.Windows.x64' + jobs: - job: ${{ parameters.jobName }} @@ -67,15 +70,14 @@ jobs: $toolPath = New-Item -ItemType Directory "$(System.DefaultWorkingDirectory)/toolPath" | Select-Object -ExpandProperty FullName - dotnet tool install --add-source "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults" --tool-path $toolPath --version '$(PowerShellVersion)' PowerShell.Windows.x64 + dotnet tool install --add-source "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults" --tool-path $toolPath --version '$(PowerShellVersion)' '${{ parameters.globalToolPackageName }}' Get-ChildItem -Path $toolPath displayName: Install global tool - pwsh: | - $exeName = if ($IsWindows) { "pwsh.exe" } else { "pwsh" } - $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/$exeName" + $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/${{ parameters.globalToolExeName }}" if (-not (Test-Path $toolPath)) { @@ -93,7 +95,7 @@ jobs: $exeName = if ($IsWindows) { "pwsh.exe" } else { "pwsh" } - $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/$exeName" + $toolPath = "$(System.DefaultWorkingDirectory)/toolPath/${{ parameters.globalToolExeName }}" $versionFound = & $toolPath -c '$PSVersionTable.PSVersion.ToString()' From e72b2106a238647a429ae0fc7ba47f44bb6a7c11 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 17:44:25 -0700 Subject: [PATCH 23/45] Add scripts for release tasks --- .../azureDevOps/releasePipeline.yml | 17 ++--- .../templates/release-BuildJson.yml | 66 +++++++++++++++++++ .../templates/release-CreateGitHubDraft.yml | 23 +++++++ .../templates/release-MakeContainerPublic.yml | 16 +++++ .../release-PublishPackageMsftCom.yml | 30 +++++++++ .../templates/release-ReleaseToNuGet.yml | 20 ++++++ .../release-SetReleaseTagAndContainerName.yml | 21 ++++++ .../release-ValidatePackageNames.yml | 24 +------ 8 files changed, 181 insertions(+), 36 deletions(-) create mode 100644 tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml create mode 100644 tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml create mode 100644 tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml create mode 100644 tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml create mode 100644 tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml create mode 100644 tools/releaseBuild/azureDevOps/templates/release-SetReleaseTagAndContainerName.yml diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 992b1343d9a..951cd5391fc 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -123,10 +123,7 @@ stages: runOnce: deploy: steps: - - download: none - - pwsh: | - Write-Host "Make azure blob public" - displayName: Make Azure Blob public" + - template: templates/release-MakeContainerPublic.yml - job: GitHubDraft displayName: Create GitHub Draft release @@ -134,9 +131,7 @@ stages: vmImage: windows-latest dependsOn: AzureBlobPublic steps: - - pwsh: | - Write-Host "Upload artifacts and create GitHub Draft Release" - displayName: Create GitHub Draft Release + - template: templates/release-CreateGitHubDraft.yml - stage: GitHubManualTasks dependsOn: GitHubDraftRelease @@ -163,17 +158,13 @@ stages: pool: vmImage: windows-latest steps: - - pwsh: | - Write-Host "Publish Nuget packages" - displayName: Publish Nuget packages + - template: templates/release-ReleaseToNuGet.yml - job: PublishPkgsMsftCom pool: vmImage: ubuntu-latest steps: - - pwsh: | - Write-Host "Publish packages.microsoft.com" - displayName: Publish to packages.microsoft.com + - template: templates/release-PublishPackageMsftCom.yml - stage: ChangesToMaster dependsOn: PublishPackages diff --git a/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml new file mode 100644 index 00000000000..895a74eaea1 --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml @@ -0,0 +1,66 @@ +steps: +- pwsh: | + $jsonFile = Get-Item "$(System.DefaultWorkingDirectory)/_Packages/BuildInfoJson/*.json" + $fileName = Split-Path $jsonFile -Leaf + + $dateTime = [datetime]::UtcNow + $dateTime = [datetime]::new($dateTime.Ticks - ($dateTime.Ticks % [timespan]::TicksPerSecond), $dateTime.Kind) + + $buildInfo = Get-Content $jsonFile | ConvertFrom-Json + $buildInfo.ReleaseDate = $dateTime + + $targetFile = "$(Agent.ReleaseDirectory)/$fileName" + ConvertTo-Json -InputObject $buildInfo | Out-File $targetFile -Encoding ascii + + $vstsCommandString = "vso[task.setvariable variable=BuildInfoJsonFile]$targetFile" + Write-Host "sending " + $vstsCommandString + Write-Host "##$vstsCommandString" + + + ## Create 'lts.json' if it's the latest stable and also a LTS release. + + $vstsCommandCopyLTSBuildInfo = "vso[task.setvariable variable=CopyLTSBuildInfo]NO" + + if ($fileName -eq "stable.json") + { + $releaseTag = $buildInfo.ReleaseTag + $version = $releaseTag -replace '^v' + $semVersion = [System.Management.Automation.SemanticVersion] $version + + if ($semVersion.PreReleaseLabel -eq $null -and $semVersion.Minor % 2 -eq 0 -and $semVersion.Major -ge 7) + { + $ltsFile = "$(Agent.ReleaseDirectory)/lts.json" + Copy-Item -Path $targetFile -Destination $ltsFile -Force + $vstsCommandLtsJsonFile = "vso[task.setvariable variable=LtsBuildInfoJsonFile]$ltsFile" + $vstsCommandCopyLTSBuildInfo = "vso[task.setvariable variable=CopyLTSBuildInfo]YES" + } + } + + Write-Host "sending " + $vstsCommandCopyLTSBuildInfo + Write-Host "##$vstsCommandCopyLTSBuildInfo" + + if ($vstsCommandLtsJsonFile) + { + Write-Host "sending " + $vstsCommandLtsJsonFile + Write-Host "##$vstsCommandLtsJsonFile" + } + displayName: Download and Capture NuPkgs + +- task: AzureFileCopy@2 + displayName: 'AzureBlob build info JSON file Copy' + inputs: + SourcePath: '$(BuildInfoJsonFile)' + azureSubscription: '$(AzureFileCopySubscription)' + Destination: AzureBlob + storage: '$(StorageAccount)' + ContainerName: BuildInfo + +- task: AzureFileCopy@2 + displayName: 'AzureBlob build info ''lts.json'' Copy when needed' + inputs: + SourcePath: '$(LtsBuildInfoJsonFile)' + azureSubscription: '$(AzureFileCopySubscription)' + Destination: AzureBlob + storage: '$(StorageAccount)' + ContainerName: BuildInfo + condition: and(succeeded(), eq(variables['CopyLTSBuildInfo'], 'YES')) diff --git a/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml b/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml new file mode 100644 index 00000000000..9bf3a9308bb --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml @@ -0,0 +1,23 @@ +steps: +- download: none + +- template: templates/release-SetReleaseTagAndContainerName.yml + +- pwsh: | + $azcopy = "C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy\AzCopy.exe" + & $azcopy /Source:https://$(StorageAccount).blob.core.windows.net/$(AzureVersion) /Dest:$(System.ArtifactsDirectory) /S /SourceKey:$(StorageAccountKey) + displayName: Download Azure Artifacts + +- pwsh: | + Get-ChildItem $(System.ArtifactsDirectory)\* -recurse | Select-Object -ExpandProperty Name + displayName: Capture downloaded artifacts + +- pwsh: | + git clone https://$(AzureDevOpsPat)@mscodehub.visualstudio.com/PowerShellCore/_git/Internal-PowerShellTeam-Tools ./tools + displayName: Clone Internal-Tools repository + +- pwsh: | + Import-module ./tools/Scripts/GitHubRelease.psm1 + Publish-ReleaseDraft -Tag '$(ReleaseTag)' -Name '$(ReleaseTag) Release of PowerShell' -Description '<-- Update Me -->' -User PowerShell -Repository PowerShell -PackageFolder $(System.ArtifactsDirectory) -Token $(GitHubReleasePat) + displayName: Publish Release Draft + diff --git a/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml b/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml new file mode 100644 index 00000000000..d3d6bc49243 --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml @@ -0,0 +1,16 @@ +steps: +- download: none + +- template: templates/release-SetReleaseTagAndContainerName.yml + +- pwsh: | + az login --service-principal -u $(az_url) -p $(az_key) --tenant $(az_name) + displayName: az login + +- pwsh: | + az storage container set-permission --account-name $(StorageAccount) --name $(azureVersion) --public-access blob + displayName: Make container public + +- pwsh: | + az logout + displayName: az logout diff --git a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml new file mode 100644 index 00000000000..4bdbab16f73 --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml @@ -0,0 +1,30 @@ +steps: +- pwsh: | + $variable = 'releaseTag' + $branch = '$(Release.Artifacts._Packages.SourceBranch)' + if($branch -notmatch '^.*(release[-/])') + { + throw "Branch name is not in release format: '$branch'" + } + + $releaseTag = $Branch -replace '^.*(release[-/])' + $vstsCommandString = "vso[task.setvariable variable=$Variable]$releaseTag" + Write-Verbose -Message "setting $Variable to $releaseTag" -Verbose + Write-Host -Object "##$vstsCommandString" + displayName: Set Release Tag + +- pwsh: | + $packageVersion = '$(ReleaseTag)'.ToLowerInvariant() -replace '^v','' + $vstsCommandString = "vso[task.setvariable variable=packageVersion]$packageVersion" + Write-Host "sending " + $vstsCommandString + Write-Host "##$vstsCommandString" + displayName: Set Package version + +- pwsh: | + git clone https://$(AzureDevOpsPat)@mscodehub.visualstudio.com/PowerShellCore/_git/Internal-PowerShellTeam-Tools ./tools + displayName: Clone Internal-PowerShellTeam-Tools from MSCodeHub + +- pwsh: | + $metadata = Get-Content -Path "_Packages/metadata/release.json" -Raw | ConvertFrom-Json + ./tools/packages.microsoft.com/upload.ps1 -ReleaseTag "$(releaseTag)" -Secret "$(PackagesRepoPassword)" -BlobFolderName "$(AzureVersion)" -LTS:($metadata.LTSRelease) + displayName: Run release script diff --git a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml new file mode 100644 index 00000000000..46816121553 --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml @@ -0,0 +1,20 @@ +steps: +- pwsh: | + #Exclude all global tool packages. Their names start with 'PowerShell.' + Copy-Item "_Packages/finalResults/*.nupkg" -Destination "$(Agent.ReleaseDirectory)" -Exclude "PowerShell.*.nupkg" -Force + + $releaseVersion = Get-Content "_Packages/metadata/release.json" | ConvertFrom-Json | Select-Object -ExpandProperty 'ReleaseVersion' + $globalToolPath = "_Packages/finalResults/PowerShell.$releaseVersion.nupkg" + ### -WhatIf to make sure we do not release global tool. Remove -WhatIf when the PowerShell name reservation is done. + Copy-Item $globalToolPath -Destination "$(Agent.ReleaseDirectory)" -WhatIf + + Get-ChildItem "$(Agent.ReleaseDirectory)" + displayName: Download and capture nupkgs + +- task: NuGetCommand@2 + displayName: 'NuGet push' + inputs: + command: push + packagesToPush: '$(Agent.ReleaseDirectory)/*.nupkg' + nuGetFeedType: external + publishFeedCredentials: PowerShellNuGetOrgPush diff --git a/tools/releaseBuild/azureDevOps/templates/release-SetReleaseTagAndContainerName.yml b/tools/releaseBuild/azureDevOps/templates/release-SetReleaseTagAndContainerName.yml new file mode 100644 index 00000000000..a1af9a4acdf --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-SetReleaseTagAndContainerName.yml @@ -0,0 +1,21 @@ +steps: +- pwsh: | + $variable = 'releaseTag' + $branch = $ENV:RESOURCES_PIPELINE_RELEASEPIPELINE_SOURCEBRANCH + if($branch -notmatch '^.*(release[-/])') + { + throw "Branch name is not in release format: '$branch'" + } + + $releaseTag = $Branch -replace '^.*(release[-/])' + $vstsCommandString = "vso[task.setvariable variable=$Variable]$releaseTag" + Write-Verbose -Message "setting $Variable to $releaseTag" -Verbose + Write-Host -Object "##$vstsCommandString" + displayName: Set Release Tag + +- pwsh: | + $azureVersion = '$(ReleaseTag)'.ToLowerInvariant() -replace '\.', '-' + $vstsCommandString = "vso[task.setvariable variable=AzureVersion]$azureVersion" + Write-Host "sending " + $vstsCommandString + Write-Host "##$vstsCommandString" + displayName: Set container name diff --git a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml index 31788609e11..a1b43fcd20c 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml @@ -3,22 +3,7 @@ steps: Get-ChildItem ENV: displayName: Capture environment -- pwsh: | - $variable = 'releaseTag' - $branch = $ENV:RESOURCES_PIPELINE_RELEASEPIPELINE_SOURCEBRANCH - - Write-Verbose -Verbose "SourceBranch == $branch" - - if ($branch -notmatch '^.*(release[-/])') { - throw "Branch name is not in release format: '$branch'" - } - - $releaseTag = $Branch -replace '^.*(release[-/])' - $vstsCommandString = "vso[task.setvariable variable=$Variable]$releaseTag" - - Write-Verbose -Message "setting $Variable to $releaseTag" -Verbose - Write-Host -Object "##$vstsCommandString" - displayName: Set Release Tag +- template: templates/release-SetReleaseTagAndContainerName.yml - pwsh: | $name = "{0}_{1:x}" -f '$(releaseTag)', (Get-Date).Ticks @@ -26,13 +11,6 @@ steps: Write-Host "##vso[build.updatebuildnumber]$name" displayName: Set Release Name -- pwsh: | - $azureVersion = '$(ReleaseTag)'.ToLowerInvariant() -replace '\.', '-' - $vstsCommandString = "vso[task.setvariable variable=AzureVersion]$azureVersion" - Write-Host "sending " + $vstsCommandString - Write-Host "##$vstsCommandString" - displayName: Set Container Name - - pwsh: | $azcopy = "C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy\AzCopy.exe" From e6702c5a188c9a15d139d3e3698510f23542b0c7 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 17:46:20 -0700 Subject: [PATCH 24/45] Fix template paths --- .../azureDevOps/templates/release-CreateGitHubDraft.yml | 2 +- .../azureDevOps/templates/release-MakeContainerPublic.yml | 2 +- .../azureDevOps/templates/release-ValidatePackageNames.yml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml b/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml index 9bf3a9308bb..647887d5332 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml @@ -1,7 +1,7 @@ steps: - download: none -- template: templates/release-SetReleaseTagAndContainerName.yml +- template: release-SetReleaseTagAndContainerName.yml - pwsh: | $azcopy = "C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy\AzCopy.exe" diff --git a/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml b/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml index d3d6bc49243..92f02cef212 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml @@ -1,7 +1,7 @@ steps: - download: none -- template: templates/release-SetReleaseTagAndContainerName.yml +- template: release-SetReleaseTagAndContainerName.yml - pwsh: | az login --service-principal -u $(az_url) -p $(az_key) --tenant $(az_name) diff --git a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml index a1b43fcd20c..d0276ab11fe 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ValidatePackageNames.yml @@ -3,7 +3,7 @@ steps: Get-ChildItem ENV: displayName: Capture environment -- template: templates/release-SetReleaseTagAndContainerName.yml +- template: release-SetReleaseTagAndContainerName.yml - pwsh: | $name = "{0}_{1:x}" -f '$(releaseTag)', (Get-Date).Ticks From f7d97e82b220350c1641a13506d80d885ed36040 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 17:47:28 -0700 Subject: [PATCH 25/45] Fix tabbing --- .../azureDevOps/templates/release-ReleaseToNuGet.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml index 46816121553..54fb060ff4b 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml @@ -13,7 +13,7 @@ steps: - task: NuGetCommand@2 displayName: 'NuGet push' - inputs: + inputs: command: push packagesToPush: '$(Agent.ReleaseDirectory)/*.nupkg' nuGetFeedType: external From ae7574e69f2cbba6349e5efec347613d84b0692a Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 17:55:02 -0700 Subject: [PATCH 26/45] Fix paths after download --- .../templates/release-BuildJson.yml | 5 ++++- .../release-PublishPackageMsftCom.yml | 19 +++++-------------- .../templates/release-ReleaseToNuGet.yml | 8 ++++++-- 3 files changed, 15 insertions(+), 17 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml index 895a74eaea1..40e2c099d4d 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml @@ -1,6 +1,9 @@ steps: +- download: releasePipeline + artifact: BuildInfoJson + - pwsh: | - $jsonFile = Get-Item "$(System.DefaultWorkingDirectory)/_Packages/BuildInfoJson/*.json" + $jsonFile = Get-Item "$ENV:PIPELINE_WORKSPACE/releasePipeline/BuildInfoJson/*.json" $fileName = Split-Path $jsonFile -Leaf $dateTime = [datetime]::UtcNow diff --git a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml index 4bdbab16f73..9d42c5ab8e9 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml @@ -1,17 +1,5 @@ steps: -- pwsh: | - $variable = 'releaseTag' - $branch = '$(Release.Artifacts._Packages.SourceBranch)' - if($branch -notmatch '^.*(release[-/])') - { - throw "Branch name is not in release format: '$branch'" - } - - $releaseTag = $Branch -replace '^.*(release[-/])' - $vstsCommandString = "vso[task.setvariable variable=$Variable]$releaseTag" - Write-Verbose -Message "setting $Variable to $releaseTag" -Verbose - Write-Host -Object "##$vstsCommandString" - displayName: Set Release Tag +- template: SetReleaseTagAndContainerName.yml - pwsh: | $packageVersion = '$(ReleaseTag)'.ToLowerInvariant() -replace '^v','' @@ -24,7 +12,10 @@ steps: git clone https://$(AzureDevOpsPat)@mscodehub.visualstudio.com/PowerShellCore/_git/Internal-PowerShellTeam-Tools ./tools displayName: Clone Internal-PowerShellTeam-Tools from MSCodeHub +- download: releasePipeline + artifact: metadata + - pwsh: | - $metadata = Get-Content -Path "_Packages/metadata/release.json" -Raw | ConvertFrom-Json + $metadata = Get-Content -Path "$ENV:PIPELINE_WORKSPACE/releasePipeline/metadata/release.json" -Raw | ConvertFrom-Json ./tools/packages.microsoft.com/upload.ps1 -ReleaseTag "$(releaseTag)" -Secret "$(PackagesRepoPassword)" -BlobFolderName "$(AzureVersion)" -LTS:($metadata.LTSRelease) displayName: Run release script diff --git a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml index 54fb060ff4b..b4a148b9ed1 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml @@ -1,10 +1,14 @@ steps: +- download: releasePipeline + artifact: finalResults + patterns: "**/*.nupkg" + - pwsh: | #Exclude all global tool packages. Their names start with 'PowerShell.' - Copy-Item "_Packages/finalResults/*.nupkg" -Destination "$(Agent.ReleaseDirectory)" -Exclude "PowerShell.*.nupkg" -Force + Copy-Item "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults/*.nupkg" -Destination "$(Agent.ReleaseDirectory)" -Exclude "PowerShell.*.nupkg" -Force $releaseVersion = Get-Content "_Packages/metadata/release.json" | ConvertFrom-Json | Select-Object -ExpandProperty 'ReleaseVersion' - $globalToolPath = "_Packages/finalResults/PowerShell.$releaseVersion.nupkg" + $globalToolPath = "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults/PowerShell.$releaseVersion.nupkg" ### -WhatIf to make sure we do not release global tool. Remove -WhatIf when the PowerShell name reservation is done. Copy-Item $globalToolPath -Destination "$(Agent.ReleaseDirectory)" -WhatIf From 7a274b0bfd5714cb822d7415fd08011079aee5bb Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 22 Jul 2020 17:56:06 -0700 Subject: [PATCH 27/45] Fix yaml template name --- .../azureDevOps/templates/release-PublishPackageMsftCom.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml index 9d42c5ab8e9..e033f94b9c6 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml @@ -1,5 +1,5 @@ steps: -- template: SetReleaseTagAndContainerName.yml +- template: release-SetReleaseTagAndContainerName.yml - pwsh: | $packageVersion = '$(ReleaseTag)'.ToLowerInvariant() -replace '^v','' From 757fbeed892e93ea577115350fd86356ce547107 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Thu, 23 Jul 2020 14:12:38 -0700 Subject: [PATCH 28/45] Enable trigger --- .../azureDevOps/releasePipeline.yml | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 951cd5391fc..13ab3520474 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -2,16 +2,15 @@ trigger: none resources: pipelines: - #- pipeline: releasePipeline - # source: Co-ordinated Build Test - # trigger: - # branches: - # - release/* - # - releaseYml - - pipeline: releasePipeline - source: 'Coordinated Packages' - branch: 'release/v7.1.0-preview.5' + source: Co-ordinated Build Test + trigger: + branches: + - release/* + + #- pipeline: releasePipeline + # source: 'Coordinated Packages' + # branch: 'release/v7.1.0-preview.5' stages: - stage: ValidateSDK From 7c6ddf3ebaf63354adf18f28004b10e3e5f50599 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Thu, 23 Jul 2020 14:48:11 -0700 Subject: [PATCH 29/45] Disable deploy tasks for testing --- .../templates/release-BuildJson.yml | 34 +++++++++---------- .../release-PublishPackageMsftCom.yml | 3 +- .../templates/release-ReleaseToNuGet.yml | 14 ++++---- 3 files changed, 26 insertions(+), 25 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml index 40e2c099d4d..dc763ce095b 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml @@ -49,21 +49,21 @@ steps: } displayName: Download and Capture NuPkgs -- task: AzureFileCopy@2 - displayName: 'AzureBlob build info JSON file Copy' - inputs: - SourcePath: '$(BuildInfoJsonFile)' - azureSubscription: '$(AzureFileCopySubscription)' - Destination: AzureBlob - storage: '$(StorageAccount)' - ContainerName: BuildInfo +#- task: AzureFileCopy@2 +# displayName: 'AzureBlob build info JSON file Copy' +# inputs: +# SourcePath: '$(BuildInfoJsonFile)' +# azureSubscription: '$(AzureFileCopySubscription)' +# Destination: AzureBlob +# storage: '$(StorageAccount)' +# ContainerName: BuildInfo -- task: AzureFileCopy@2 - displayName: 'AzureBlob build info ''lts.json'' Copy when needed' - inputs: - SourcePath: '$(LtsBuildInfoJsonFile)' - azureSubscription: '$(AzureFileCopySubscription)' - Destination: AzureBlob - storage: '$(StorageAccount)' - ContainerName: BuildInfo - condition: and(succeeded(), eq(variables['CopyLTSBuildInfo'], 'YES')) +#- task: AzureFileCopy@2 +# displayName: 'AzureBlob build info ''lts.json'' Copy when needed' +# inputs: +# SourcePath: '$(LtsBuildInfoJsonFile)' +# azureSubscription: '$(AzureFileCopySubscription)' +# Destination: AzureBlob +# storage: '$(StorageAccount)' +# ContainerName: BuildInfo +# condition: and(succeeded(), eq(variables['CopyLTSBuildInfo'], 'YES')) diff --git a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml index e033f94b9c6..1d873429f6c 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml @@ -17,5 +17,6 @@ steps: - pwsh: | $metadata = Get-Content -Path "$ENV:PIPELINE_WORKSPACE/releasePipeline/metadata/release.json" -Raw | ConvertFrom-Json - ./tools/packages.microsoft.com/upload.ps1 -ReleaseTag "$(releaseTag)" -Secret "$(PackagesRepoPassword)" -BlobFolderName "$(AzureVersion)" -LTS:($metadata.LTSRelease) + Write-Vebose -Verbose "$(releaseTag) $(PackagesRepoPassword) $(AzureVersion) $($metadata.LTSRelease)" + #./tools/packages.microsoft.com/upload.ps1 -ReleaseTag "$(releaseTag)" -Secret "$(PackagesRepoPassword)" -BlobFolderName "$(AzureVersion)" -LTS:($metadata.LTSRelease) displayName: Run release script diff --git a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml index b4a148b9ed1..8fa4e1857ed 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml @@ -15,10 +15,10 @@ steps: Get-ChildItem "$(Agent.ReleaseDirectory)" displayName: Download and capture nupkgs -- task: NuGetCommand@2 - displayName: 'NuGet push' - inputs: - command: push - packagesToPush: '$(Agent.ReleaseDirectory)/*.nupkg' - nuGetFeedType: external - publishFeedCredentials: PowerShellNuGetOrgPush +#- task: NuGetCommand@2 +# displayName: 'NuGet push' +# inputs: +# command: push +# packagesToPush: '$(Agent.ReleaseDirectory)/*.nupkg' +# nuGetFeedType: external +# publishFeedCredentials: PowerShellNuGetOrgPush From a53070fe4af68ef4abbb308561893050c8eb7cba Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Thu, 23 Jul 2020 17:26:59 -0700 Subject: [PATCH 30/45] Fix yaml --- .../azureDevOps/templates/release-MakeContainerPublic.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml b/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml index 92f02cef212..870f2794dcb 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-MakeContainerPublic.yml @@ -5,12 +5,12 @@ steps: - pwsh: | az login --service-principal -u $(az_url) -p $(az_key) --tenant $(az_name) - displayName: az login + displayName: az login - pwsh: | az storage container set-permission --account-name $(StorageAccount) --name $(azureVersion) --public-access blob - displayName: Make container public + displayName: Make container public - pwsh: | az logout - displayName: az logout + displayName: az logout From 32e745472c80d15aefe89f2399189f595b002638 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Fri, 24 Jul 2020 14:06:32 -0700 Subject: [PATCH 31/45] Fix local clone of internal repo --- .../azureDevOps/templates/release-CreateGitHubDraft.yml | 5 ++--- .../azureDevOps/templates/release-PublishPackageMsftCom.yml | 4 ++-- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml b/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml index 647887d5332..3b316e84cbf 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-CreateGitHubDraft.yml @@ -13,11 +13,10 @@ steps: displayName: Capture downloaded artifacts - pwsh: | - git clone https://$(AzureDevOpsPat)@mscodehub.visualstudio.com/PowerShellCore/_git/Internal-PowerShellTeam-Tools ./tools + git clone https://$(AzureDevOpsPat)@mscodehub.visualstudio.com/PowerShellCore/_git/Internal-PowerShellTeam-Tools '$(Pipeline.Workspace)/tools' displayName: Clone Internal-Tools repository - pwsh: | - Import-module ./tools/Scripts/GitHubRelease.psm1 + Import-module '$(Pipeline.Workspace)/tools/Scripts/GitHubRelease.psm1' Publish-ReleaseDraft -Tag '$(ReleaseTag)' -Name '$(ReleaseTag) Release of PowerShell' -Description '<-- Update Me -->' -User PowerShell -Repository PowerShell -PackageFolder $(System.ArtifactsDirectory) -Token $(GitHubReleasePat) displayName: Publish Release Draft - diff --git a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml index 1d873429f6c..6f1eccc6e8c 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml @@ -9,7 +9,7 @@ steps: displayName: Set Package version - pwsh: | - git clone https://$(AzureDevOpsPat)@mscodehub.visualstudio.com/PowerShellCore/_git/Internal-PowerShellTeam-Tools ./tools + git clone https://$(AzureDevOpsPat)@mscodehub.visualstudio.com/PowerShellCore/_git/Internal-PowerShellTeam-Tools '$(Pipeline.Workspace)/tools' displayName: Clone Internal-PowerShellTeam-Tools from MSCodeHub - download: releasePipeline @@ -18,5 +18,5 @@ steps: - pwsh: | $metadata = Get-Content -Path "$ENV:PIPELINE_WORKSPACE/releasePipeline/metadata/release.json" -Raw | ConvertFrom-Json Write-Vebose -Verbose "$(releaseTag) $(PackagesRepoPassword) $(AzureVersion) $($metadata.LTSRelease)" - #./tools/packages.microsoft.com/upload.ps1 -ReleaseTag "$(releaseTag)" -Secret "$(PackagesRepoPassword)" -BlobFolderName "$(AzureVersion)" -LTS:($metadata.LTSRelease) + & '$(Pipeline.Workspace)/tools/packages.microsoft.com/upload.ps1' -ReleaseTag "$(releaseTag)" -Secret "$(PackagesRepoPassword)" -BlobFolderName "$(AzureVersion)" -LTS:($metadata.LTSRelease) displayName: Run release script From bcde4f1117dfef9c5d90f1afdc26b3124dfa484f Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Fri, 24 Jul 2020 14:55:09 -0700 Subject: [PATCH 32/45] Fix issues in package release --- .../templates/release-PublishPackageMsftCom.yml | 2 +- .../azureDevOps/templates/release-ReleaseToNuGet.yml | 11 +++++++---- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml index 6f1eccc6e8c..27d76a11fed 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-PublishPackageMsftCom.yml @@ -17,6 +17,6 @@ steps: - pwsh: | $metadata = Get-Content -Path "$ENV:PIPELINE_WORKSPACE/releasePipeline/metadata/release.json" -Raw | ConvertFrom-Json - Write-Vebose -Verbose "$(releaseTag) $(PackagesRepoPassword) $(AzureVersion) $($metadata.LTSRelease)" + Write-Verbose -Verbose "$(releaseTag) $(PackagesRepoPassword) $(AzureVersion) $($metadata.LTSRelease)" & '$(Pipeline.Workspace)/tools/packages.microsoft.com/upload.ps1' -ReleaseTag "$(releaseTag)" -Secret "$(PackagesRepoPassword)" -BlobFolderName "$(AzureVersion)" -LTS:($metadata.LTSRelease) displayName: Run release script diff --git a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml index 8fa4e1857ed..225de2c0136 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml @@ -3,16 +3,19 @@ steps: artifact: finalResults patterns: "**/*.nupkg" +- download: releasePipeline + artifact: metadata + - pwsh: | #Exclude all global tool packages. Their names start with 'PowerShell.' - Copy-Item "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults/*.nupkg" -Destination "$(Agent.ReleaseDirectory)" -Exclude "PowerShell.*.nupkg" -Force + Copy-Item "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults/*.nupkg" -Destination "$(Pipeline.Workspace)/release" -Exclude "PowerShell.*.nupkg" -Force - $releaseVersion = Get-Content "_Packages/metadata/release.json" | ConvertFrom-Json | Select-Object -ExpandProperty 'ReleaseVersion' + $releaseVersion = Get-Content "$ENV:PIPELINE_WORKSPACE/releasePipeline/metadata/release.json" | ConvertFrom-Json | Select-Object -ExpandProperty 'ReleaseVersion' $globalToolPath = "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults/PowerShell.$releaseVersion.nupkg" ### -WhatIf to make sure we do not release global tool. Remove -WhatIf when the PowerShell name reservation is done. - Copy-Item $globalToolPath -Destination "$(Agent.ReleaseDirectory)" -WhatIf + Copy-Item $globalToolPath -Destination "$(Pipeline.Workspace)/release" -WhatIf - Get-ChildItem "$(Agent.ReleaseDirectory)" + Get-ChildItem "$(Pipeline.Workspace)" displayName: Download and capture nupkgs #- task: NuGetCommand@2 From 46dee0de31b79f8b589b8058e3449b81d8faa1bf Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Fri, 24 Jul 2020 15:08:26 -0700 Subject: [PATCH 33/45] Fix issues in buildjson --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 4 +--- .../releaseBuild/azureDevOps/templates/release-BuildJson.yml | 4 ++-- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 13ab3520474..9b03fbccd97 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -205,9 +205,7 @@ stages: pool: vmImage: windows-latest steps: - - pwsh: | - Write-Host "Upload build json" - displayName: Upload Build JSON for upgrade notification + - templates: template/release-BuildJson.yml - stage: ReleaseVPack dependsOn: GitHubManualTasks diff --git a/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml index dc763ce095b..09416c339b1 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml @@ -12,7 +12,7 @@ steps: $buildInfo = Get-Content $jsonFile | ConvertFrom-Json $buildInfo.ReleaseDate = $dateTime - $targetFile = "$(Agent.ReleaseDirectory)/$fileName" + $targetFile = "$ENV:PIPELINE_WORKSPACE/$fileName" ConvertTo-Json -InputObject $buildInfo | Out-File $targetFile -Encoding ascii $vstsCommandString = "vso[task.setvariable variable=BuildInfoJsonFile]$targetFile" @@ -32,7 +32,7 @@ steps: if ($semVersion.PreReleaseLabel -eq $null -and $semVersion.Minor % 2 -eq 0 -and $semVersion.Major -ge 7) { - $ltsFile = "$(Agent.ReleaseDirectory)/lts.json" + $ltsFile = "$ENV:PIPELINE_WORKSPACE/lts.json" Copy-Item -Path $targetFile -Destination $ltsFile -Force $vstsCommandLtsJsonFile = "vso[task.setvariable variable=LtsBuildInfoJsonFile]$ltsFile" $vstsCommandCopyLTSBuildInfo = "vso[task.setvariable variable=CopyLTSBuildInfo]YES" From 6be6386f08bc98fb963038da69dcc0c42c635cda Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Fri, 24 Jul 2020 15:09:41 -0700 Subject: [PATCH 34/45] Fix typo --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 9b03fbccd97..014062a1e41 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -205,7 +205,7 @@ stages: pool: vmImage: windows-latest steps: - - templates: template/release-BuildJson.yml + - template: template/release-BuildJson.yml - stage: ReleaseVPack dependsOn: GitHubManualTasks From 37cdb2b548d82061c67be7ee323dbf7ca60092ec Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Fri, 24 Jul 2020 15:14:00 -0700 Subject: [PATCH 35/45] Fix typo --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 014062a1e41..23c0755ea91 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -205,7 +205,7 @@ stages: pool: vmImage: windows-latest steps: - - template: template/release-BuildJson.yml + - template: templates/release-BuildJson.yml - stage: ReleaseVPack dependsOn: GitHubManualTasks From ceabead6ac70acd582b4673963d3c6cbd0d8d031 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Mon, 10 Aug 2020 14:30:03 -0700 Subject: [PATCH 36/45] Add approval for VPack and remove commented publish code --- .../azureDevOps/releasePipeline.yml | 11 +++--- .../templates/release-BuildJson.yml | 34 +++++++++---------- .../templates/release-GlobalToolTest.yml | 22 ++++++------ .../templates/release-ReleaseToNuGet.yml | 14 ++++---- 4 files changed, 39 insertions(+), 42 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 23c0755ea91..958d57f6ec2 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -211,13 +211,10 @@ stages: dependsOn: GitHubManualTasks displayName: Release VPack jobs: - - job: VPackRelease - pool: - vmImage: windows-latest - steps: - - pwsh: | - Write-Host "Release VPack" - displayName: Release VPack + - deployment: ReleaseVPackManual + displayName: Release VPack + pool: server + environment: PSReleaseVpack - stage: ReleaseClose displayName: Finish Release diff --git a/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml index 09416c339b1..8def847dce2 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-BuildJson.yml @@ -49,21 +49,21 @@ steps: } displayName: Download and Capture NuPkgs -#- task: AzureFileCopy@2 -# displayName: 'AzureBlob build info JSON file Copy' -# inputs: -# SourcePath: '$(BuildInfoJsonFile)' -# azureSubscription: '$(AzureFileCopySubscription)' -# Destination: AzureBlob -# storage: '$(StorageAccount)' -# ContainerName: BuildInfo +- task: AzureFileCopy@2 + displayName: 'AzureBlob build info JSON file Copy' + inputs: + SourcePath: '$(BuildInfoJsonFile)' + azureSubscription: '$(AzureFileCopySubscription)' + Destination: AzureBlob + storage: '$(StorageAccount)' + ContainerName: BuildInfo -#- task: AzureFileCopy@2 -# displayName: 'AzureBlob build info ''lts.json'' Copy when needed' -# inputs: -# SourcePath: '$(LtsBuildInfoJsonFile)' -# azureSubscription: '$(AzureFileCopySubscription)' -# Destination: AzureBlob -# storage: '$(StorageAccount)' -# ContainerName: BuildInfo -# condition: and(succeeded(), eq(variables['CopyLTSBuildInfo'], 'YES')) +- task: AzureFileCopy@2 + displayName: 'AzureBlob build info ''lts.json'' Copy when needed' + inputs: + SourcePath: '$(LtsBuildInfoJsonFile)' + azureSubscription: '$(AzureFileCopySubscription)' + Destination: AzureBlob + storage: '$(StorageAccount)' + ContainerName: BuildInfo + condition: and(succeeded(), eq(variables['CopyLTSBuildInfo'], 'YES')) diff --git a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml index 795bc30b00b..7d9a49189bd 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-GlobalToolTest.yml @@ -16,17 +16,17 @@ jobs: artifact: finalResults patterns: '**/*.nupkg' - #- task: DownloadBuildArtifacts@0 - # inputs: - # buildType: specific - # project: PowerShellCore - # pipeline: Co-ordinated Build Test - # specificBuildWithTriggering: true - # buildVersionToDownload: latest - # downloadType: single - # artifactName: finalResults - # downloadPath: '$(Pipeline.Workspace)' - # itemPattern: '*.nupkg' + - task: DownloadBuildArtifacts@0 + inputs: + buildType: specific + project: PowerShellCore + pipeline: Co-ordinated Build Test + specificBuildWithTriggering: true + buildVersionToDownload: latest + downloadType: single + artifactName: finalResults + downloadPath: '$(Pipeline.Workspace)' + itemPattern: '*.nupkg' - pwsh: | $dotnetMetadataPath = "$(Build.SourcesDirectory)/DotnetRuntimeMetadata.json" diff --git a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml index 225de2c0136..dd8404f5dce 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-ReleaseToNuGet.yml @@ -18,10 +18,10 @@ steps: Get-ChildItem "$(Pipeline.Workspace)" displayName: Download and capture nupkgs -#- task: NuGetCommand@2 -# displayName: 'NuGet push' -# inputs: -# command: push -# packagesToPush: '$(Agent.ReleaseDirectory)/*.nupkg' -# nuGetFeedType: external -# publishFeedCredentials: PowerShellNuGetOrgPush +- task: NuGetCommand@2 + displayName: 'NuGet push' + inputs: + command: push + packagesToPush: '$(Agent.ReleaseDirectory)/*.nupkg' + nuGetFeedType: external + publishFeedCredentials: PowerShellNuGetOrgPush From 5df77cec679ee14fdeedb90ee046eace81c61249 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Mon, 10 Aug 2020 14:57:50 -0700 Subject: [PATCH 37/45] Fix build name --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 958d57f6ec2..30f36d6391c 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -3,15 +3,11 @@ trigger: none resources: pipelines: - pipeline: releasePipeline - source: Co-ordinated Build Test + source: 'Coordinated Packages' trigger: branches: - release/* - #- pipeline: releasePipeline - # source: 'Coordinated Packages' - # branch: 'release/v7.1.0-preview.5' - stages: - stage: ValidateSDK dependsOn: [] From 1cc6599baa5d873c2e344dbbfeec763d7733f4ac Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Mon, 10 Aug 2020 15:44:19 -0700 Subject: [PATCH 38/45] Undo change for disabling compliance --- tools/releaseBuild/azureDevOps/releaseBuild.yml | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releaseBuild.yml b/tools/releaseBuild/azureDevOps/releaseBuild.yml index 9c509099187..803dd01c90f 100644 --- a/tools/releaseBuild/azureDevOps/releaseBuild.yml +++ b/tools/releaseBuild/azureDevOps/releaseBuild.yml @@ -119,13 +119,13 @@ jobs: - template: templates/mac-package-signing.yml -#- template: templates/compliance.yml -# parameters: -# parentJobs: -# - build_windows_x64 -# - build_windows_x86 -# - build_windows_fxdependent -# - build_windows_fxdependentWinDesktop +- template: templates/compliance.yml + parameters: + parentJobs: + - build_windows_x64 + - build_windows_x86 + - build_windows_fxdependent + - build_windows_fxdependentWinDesktop - template: templates/nuget.yml parameters: From fd5c9ac27030da66cd190dce79eca8cf33ad5099 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Tue, 11 Aug 2020 10:22:39 -0700 Subject: [PATCH 39/45] Add update deps step --- .../releaseBuild/azureDevOps/releasePipeline.yml | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 30f36d6391c..dbf6de3048b 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -212,6 +212,21 @@ stages: pool: server environment: PSReleaseVpack +- stage: ReleaseDeps + dependsOn: GitHubManualTasks + displayName: Update pwsh.deps.json links + jobs: + - job: UpdateDepsFiles + displayName: Update deps files + pool: + vmImage: windows-latest + steps: + - download: releasePipeline + artifact: finalResults + patterns: '**/PowerShell*-win-x64.zip' + + + - stage: ReleaseClose displayName: Finish Release dependsOn: From 05d02e49cace8d6ba104d0cbd5e1d750cfc8f7e3 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Tue, 11 Aug 2020 10:54:47 -0700 Subject: [PATCH 40/45] Empty commit From d020e60ee034a8e855722ded4971caa177749e05 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Tue, 11 Aug 2020 17:24:22 -0700 Subject: [PATCH 41/45] Add release deps json to pipeline --- .../azureDevOps/releasePipeline.yml | 21 +++++----- .../templates/release-UpdateDepsJson.yml | 41 +++++++++++++++++++ 2 files changed, 52 insertions(+), 10 deletions(-) create mode 100644 tools/releaseBuild/azureDevOps/templates/release-UpdateDepsJson.yml diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index dbf6de3048b..139e71dec2a 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -1,3 +1,7 @@ +# needed to disable CI trigger and allow manual trigger +# when the branch is same as pipline source, the latest build from the source is used. +# all environment used are for manual tasks and approvals. + trigger: none resources: @@ -8,6 +12,9 @@ resources: branches: - release/* +variables: + runCodesignValidationInjection: false + stages: - stage: ValidateSDK dependsOn: [] @@ -100,6 +107,7 @@ stages: displayName: Start Release Automation - stage: GitHubDraftRelease + # do not include stages that are likely to fail in dependency as there is no way to force deploy. dependsOn: - ValidateSDK - PRCreation @@ -108,6 +116,7 @@ stages: - ManualValidation - ReleaseAutomation +# The environment here is used for approval. jobs: - deployment: AzureBlobPublic displayName: Make Azure Blob Public @@ -216,16 +225,7 @@ stages: dependsOn: GitHubManualTasks displayName: Update pwsh.deps.json links jobs: - - job: UpdateDepsFiles - displayName: Update deps files - pool: - vmImage: windows-latest - steps: - - download: releasePipeline - artifact: finalResults - patterns: '**/PowerShell*-win-x64.zip' - - + - template: templates/release-UpdateDepsJson.yml - stage: ReleaseClose displayName: Finish Release @@ -235,6 +235,7 @@ stages: - UpdateDotnetDocker - ReleaseSnapAndDocker - ChangesToMaster + - ReleaseDeps jobs: - deployment: RetainBuild displayName: Retain Build diff --git a/tools/releaseBuild/azureDevOps/templates/release-UpdateDepsJson.yml b/tools/releaseBuild/azureDevOps/templates/release-UpdateDepsJson.yml new file mode 100644 index 00000000000..51e32e30ec8 --- /dev/null +++ b/tools/releaseBuild/azureDevOps/templates/release-UpdateDepsJson.yml @@ -0,0 +1,41 @@ +jobs: +- job: UpdateDepsFiles + displayName: Update deps files + pool: + vmImage: windows-latest + steps: + - download: releasePipeline + artifact: finalResults + patterns: '**/PowerShell*-win-x64.zip' + + - download: releasePipeline + artifact: BuildInfoJson + + - pwsh: | + $fileName = (Get-Item "$ENV:PIPELINE_WORKSPACE/releasePipeline/BuildInfoJson/*.json").BaseName + if ($fileName -notin 'stable','preview') + { + throw "Unexpected fileName: $fileName" + } + + $vstsCommand = "vso[task.setvariable variable=ContainerName]$fileName" + Write-Host "##$vstsCommand" + displayName: Determine container name + + - pwsh: | + $zipFile = (Get-Item "$ENV:PIPELINE_WORKSPACE/releasePipeline/finalResults/PowerShell*-winx64.zip") + Expand-Archive -Path $zipFile -Destination "$ENV:PIPELINE_WORKSPACE/expanded" + + $pwshDepsFile = Get-Item "$ENV:PIPELINE_WORKSPACE/expanded/pwsh.deps.json" + $vstsCommand = "vso[task.setvariable variable=FileToUpload]$pwshDepsFile" + Write-Host "##$vstsCommand" + displayName: Determine file to upload + + - task: AzureFileCopy@2 + displayName: 'AzureBlob pwsh.deps.json file Copy' + inputs: + SourcePath: '$(FileToUpload)' + azureSubscription: '$(AzureFileCopySubscription)' + Destination: AzureBlob + storage: '$(StorageAccount)' + ContainerName: ps-deps-json From 2b126d3429a171723a789a5b36a4b3dd97ff8f92 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Tue, 11 Aug 2020 17:30:40 -0700 Subject: [PATCH 42/45] Address feedback --- .../azureDevOps/releasePipeline.yml | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 139e71dec2a..5b848e18e84 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -100,11 +100,10 @@ stages: - stage: ReleaseAutomation dependsOn: [] jobs: - - job: RAKickOff - steps: - - pwsh: | - Write-Host "Start release automation here" - displayName: Start Release Automation + - deployment: ReleaseAutomationKickOff + displayName: Kick off release ReleaseAutomation + pool: server + environment: PSReleaseAutomationKickOff - stage: GitHubDraftRelease # do not include stages that are likely to fail in dependency as there is no way to force deploy. @@ -178,16 +177,20 @@ stages: pool: server environment: PSReleaseChangesToMaster -- stage: ReleaseSnapAndDocker +- stage: ReleaseDocker dependsOn: - - PublishPackages - - ChangesToMaster + - StartDocker jobs: - deployment: ReleaseDocker displayName: Release Docker pool: server environment: PSReleaseDockerRelease +- stage: ReleaseSnap + dependsOn: + - PublishPackages + - ChangesToMaster + - deployment: ReleaseSnap displayName: Release Snap pool: server From edbb4e27af155b6a09ee6a09fc5085bf5a0d3080 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 12 Aug 2020 10:13:25 -0700 Subject: [PATCH 43/45] Add variable groups to jobs when using variables --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 12 ++++++++++++ .../azureDevOps/templates/release-UpdateDepsJson.yml | 2 ++ 2 files changed, 14 insertions(+) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 5b848e18e84..3d3dbb8d26a 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -71,6 +71,8 @@ stages: - job: ValidatePkgNames pool: vmImage: windows-latest + variables: + - group: 'Azure Blob variable group' steps: - template: templates/release-ValidatePackageNames.yml @@ -121,6 +123,8 @@ stages: displayName: Make Azure Blob Public pool: vmImage: windows-latest + variables: + - group: 'ACR' environment: PSReleaseAzureBlobPublic strategy: runOnce: @@ -132,6 +136,9 @@ stages: displayName: Create GitHub Draft release pool: vmImage: windows-latest + variables: + - group: 'Azure Blob variable group' + - group: 'AzDevOpsArtifacts' dependsOn: AzureBlobPublic steps: - template: templates/release-CreateGitHubDraft.yml @@ -166,6 +173,9 @@ stages: - job: PublishPkgsMsftCom pool: vmImage: ubuntu-latest + variables: + - group: 'AzDevOpsArtifacts' + - group: 'packages.microsoft.com' steps: - template: templates/release-PublishPackageMsftCom.yml @@ -212,6 +222,8 @@ stages: - job: UploadJson pool: vmImage: windows-latest + variables: + - group: 'Azure Blob variable group' steps: - template: templates/release-BuildJson.yml diff --git a/tools/releaseBuild/azureDevOps/templates/release-UpdateDepsJson.yml b/tools/releaseBuild/azureDevOps/templates/release-UpdateDepsJson.yml index 51e32e30ec8..8c2b94dae12 100644 --- a/tools/releaseBuild/azureDevOps/templates/release-UpdateDepsJson.yml +++ b/tools/releaseBuild/azureDevOps/templates/release-UpdateDepsJson.yml @@ -3,6 +3,8 @@ jobs: displayName: Update deps files pool: vmImage: windows-latest + variables: + - group: 'Azure Blob variable group' steps: - download: releasePipeline artifact: finalResults From 2f2812c75e4c320c683da236cd89df8d9df8f852 Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 12 Aug 2020 10:22:22 -0700 Subject: [PATCH 44/45] Fix yaml and add display names to stages --- .../releaseBuild/azureDevOps/releasePipeline.yml | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index 3d3dbb8d26a..a35c517a64a 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -1,9 +1,9 @@ +trigger: none + # needed to disable CI trigger and allow manual trigger # when the branch is same as pipline source, the latest build from the source is used. # all environment used are for manual tasks and approvals. -trigger: none - resources: pipelines: - pipeline: releasePipeline @@ -17,6 +17,7 @@ variables: stages: - stage: ValidateSDK + displayName: Validate SDK dependsOn: [] jobs: - template: templates/release-SDKTests.yml @@ -38,6 +39,7 @@ stages: imageName: macOS-latest - stage: PRCreation + displayName: Create PR in GH Master dependsOn: [] jobs: - deployment: CreatePRInMaster @@ -46,6 +48,7 @@ stages: environment: PSReleaseCreatePR - stage: ValidateGlobalTool + displayName: Validate Global Tool dependsOn: [] jobs: - template: templates/release-GlobalToolTest.yml @@ -100,6 +103,7 @@ stages: environment: PSReleaseLinuxPkgValidate - stage: ReleaseAutomation + displayName: Release Automation dependsOn: [] jobs: - deployment: ReleaseAutomationKickOff @@ -108,6 +112,7 @@ stages: environment: PSReleaseAutomationKickOff - stage: GitHubDraftRelease + displayName: Create GitHub draft release # do not include stages that are likely to fail in dependency as there is no way to force deploy. dependsOn: - ValidateSDK @@ -144,6 +149,7 @@ stages: - template: templates/release-CreateGitHubDraft.yml - stage: GitHubManualTasks + displayName: GitHub manual tasks dependsOn: GitHubDraftRelease jobs: - deployment: UpdateChangeLog @@ -162,6 +168,7 @@ stages: environment: PSReleaseDraftPublic - stage: PublishPackages + displayName: Publish packages dependsOn: GitHubManualTasks jobs: - job: PublishNuget @@ -180,6 +187,7 @@ stages: - template: templates/release-PublishPackageMsftCom.yml - stage: ChangesToMaster + displayName: Ensure changes are in GH master dependsOn: PublishPackages jobs: - deployment: MergeToMaster @@ -188,6 +196,7 @@ stages: environment: PSReleaseChangesToMaster - stage: ReleaseDocker + displayName: Release Docker dependsOn: - StartDocker jobs: @@ -197,10 +206,11 @@ stages: environment: PSReleaseDockerRelease - stage: ReleaseSnap + displayName: Release Snap dependsOn: - PublishPackages - ChangesToMaster - + jobs: - deployment: ReleaseSnap displayName: Release Snap pool: server From 5f2280021065b0b19c86a2071dd40018db0a896b Mon Sep 17 00:00:00 2001 From: Aditya Patwardhan Date: Wed, 12 Aug 2020 10:24:28 -0700 Subject: [PATCH 45/45] Fix stage dependencies --- tools/releaseBuild/azureDevOps/releasePipeline.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/releaseBuild/azureDevOps/releasePipeline.yml b/tools/releaseBuild/azureDevOps/releasePipeline.yml index a35c517a64a..99d9091d848 100644 --- a/tools/releaseBuild/azureDevOps/releasePipeline.yml +++ b/tools/releaseBuild/azureDevOps/releasePipeline.yml @@ -258,7 +258,8 @@ stages: - ReleaseVPack - BuildInfoJson - UpdateDotnetDocker - - ReleaseSnapAndDocker + - ReleaseDocker + - ReleaseSnap - ChangesToMaster - ReleaseDeps jobs: