Maximum compressing Azure DevOps tasks for publishing

Sometimes, when working with Azure DevOps custom Tasks, specially when the repository holds many tasks published as a single *.vsix package, its size can be a problem for publishing it.

In the official Azure documentation the maximum file size is set to 49MB, but this size it isn’t the same when you are using the CLI Microsoft provides az extension publish

Webpack to the rescue!

Webpack is used in web development for minimizing and transpiling Javascript / Typescript code, and for our requirements, we can achieve the same with some handy hacks:

First of all we need to configure some important things for node environment:

target: 'node',
  node: {
    __dirname: false,
    __filename: false,
  },

The previous snippet tells webpack to transpile considering a nodeJS environment instead of web, which is the webpack default. node __dirname and __filename configs are also required if we are going to reference them, otherwise, they will be replaced with ‘/’ instead.

Detecting Tasks as entries

entry: glob.sync("./Tasks/*/index.ts").reduce((acc, item) => {
    const path = item.split("/");
    path.pop();
    const name = path.join('/');
    acc[name] = item;
    return acc;
  }, {}),
  output: {
    path: path.resolve("dist/"),
    filename: '[name]/dist/index.js',
    libraryTarget: 'commonjs',
    devtoolModuleFilenameTemplate: '../[resource-path]'
  },

This snippet will look for every index.ts file inside Tasks directory, and then, will output the transpiled code inside ./Tasks/**/dist directory for every Task found.

Solving azure-pipelines-task-lib/task dependency

If you try to optimize this library, you will have to deal with many errors and different file types used internally by the task, the easiest solution is tho declare this dependency as external, this will force webpack to reference the ‘real’ dependency without optimizing it.

externals: {
    'azure-pipelines-task-lib/task': 'commonjs2 azure-pipelines-task-lib/task',
    ... other common libraries 
  },

Dealing with task required files

This plugin will copy all the required files for every task and then, it will copy them to the appropriate final directory

plugins: [
    new CopyPlugin({
      patterns: [
        { from: 'Tasks/*/task.json', 
          to: './' 
        },
        { from: 'Tasks/*/package.json', 
        to: './' 
        },
        { from: 'Tasks/*/*.png', 
          to: './' 
        },
        {
          from: '*.json',
          to: './'
        },
        {
          from: '*.png',
          to: './'
        },
        {
          from: 'images/*',
          to: './'
        },
        {
          from: './README.md',
          to: './'
        },
        {
          from: './license.txt',
          to: './'
        },
        { 
          from: 'Tasks/*/*.xm4', 
          to: './' 
        },
        { 
          from: 'Tasks/*/*.ps1', 
          to: './' 
        },
        { 
          from: 'Tasks/*/*.hbs', 
          to: './' 
        }
      ],
    }),

Complete webpack config

const path = require('path');
const glob = require('glob');
const CopyPlugin = require('copy-webpack-plugin');

/**@type {import('webpack').Configuration}*/
module.exports = {
  target: 'node',
  node: {
    __dirname: false,
    __filename: false,
  },
  mode: 'production',
  entry: glob.sync("./Tasks/*/index.ts").reduce((acc, item) => {
    const path = item.split("/");
    path.pop();
    const name = path.join('/');
    acc[name] = item;
    return acc;
  }, {}),
  output: {
    path: path.resolve("dist/"),
    filename: '[name]/dist/index.js',
    libraryTarget: 'commonjs',
    devtoolModuleFilenameTemplate: '../[resource-path]'
  },
  optimization: {
    runtimeChunk: false
  },
  externals: {
    'azure-pipelines-task-lib/task': 'commonjs2 azure-pipelines-task-lib/task'
  },
  plugins: [
    new CopyPlugin({
      patterns: [
        { from: 'Tasks/*/task.json', 
          to: './' 
        },
        { from: 'Tasks/*/package.json', 
        to: './' 
        },
        { from: 'Tasks/*/*.png', 
          to: './' 
        },
        {
          from: '*.json',
          to: './'
        },
        {
          from: '*.png',
          to: './'
        },
        {
          from: 'images/*',
          to: './'
        },
        {
          from: './README.md',
          to: './'
        },
        {
          from: './license.txt',
          to: './'
        },
        { 
          from: 'Tasks/*/*.xm4', 
          to: './' 
        },
        { 
          from: 'Tasks/*/*.ps1', 
          to: './' 
        },
        { 
          from: 'Tasks/*/*.hbs', 
          to: './' 
        }
      ],
    }),
  ],
  module: {
    rules: [
      {
        test: /\.ts?$/,
        exclude: /(node_modules)/,
        use: 'babel-loader',
      },
      {
        test: /\.resjson$/,
        loader: 'json-loader'
      }
    ],
  },
  resolve: {
    extensions: ['.js', '.ts', '.json', '.resjson'],
  },
};

Handling dependency fetching

As now we have two dependency levels, we have to take care of in-task dependencies resolution.

I’ve created few useful npm scripts for easing this task:

//package.json
"scripts": {
    "build": "npm run install:recursive && npm run build:webpack",
    "build:webpack": "webpack --config webpack.config.js",
    "install:recursive": "node node_modules/recursive-install/recursive-install.js --rootDir=Tasks",
    "install:distrecursive": "node node_modules/recursive-install/recursive-install.js --rootDir=dist/Tasks --production",

},
"devDependencies": {
    "recursive-install": "^1.4.0",
    "glob": "^7.1.6"
}

package.json snippet

EXTRA: Azure Pipelines

trigger:
  - master
  - feature/*
  - ci/*

variables:
  major: 0
  minor: 13
  patch: $[ counter( format('{0},{1}', variables.major, variables.minor), 0 ) ]

name: $(major).$(minor).$(patch)
  
pool:
  vmImage: 'ubuntu-latest'

steps:
- task: NodeTool@0
  inputs:
    versionSpec: '12.x'
  displayName: 'Install Node.js'

- task: npmAuthenticate@0
  inputs:
    workingFile: '.npmrc'
    customEndpoint: 'ALM npm Repository'

- bash: find ./Tasks/* -type d -exec cp .npmrc {} \;
  displayName: Propagate auth details to Tasks subprojects

- task: Npm@1
  inputs:
    command: 'install'
    customEndpoint: 'ALM npm Repository'

- script: npm run install:recursive
  displayName: 'Install npm Tasks deps.'

- script: npm run build
  displayName: Build tasks

- bash: find ./dist/Tasks/* -type d -exec cp .npmrc {} \;
  displayName: Propagate auth details to Tasks dist subprojects

- script: npm run install:distrecursive
  displayName: Install node_modules production only for dist

- task: VersionJSONFile@2
  inputs:
    Path: '$(Build.SourcesDirectory)'
    recursion: true
    VersionNumber: '$(Build.BuildNumber)'
    useBuildNumberDirectly: true
    FilenamePattern: 'package.json'
    OutputVersion: 'OutputedVersion'
- task: VersionJSONFile@2
  inputs:
    Path: '$(Build.SourcesDirectory)'
    recursion: true
    VersionNumber: '$(Build.BuildNumber)'
    useBuildNumberDirectly: true
    FilenamePattern: 'vss-extension.json'
    OutputVersion: 'OutputedVersion'

- script: npm i -g tfx-cli
  displayName: Install TFX-CLI
- script: tfx login --auth-type pat -t $(patSecret) -u https://marketplace.visualstudio.com
  displayName: TFX Login
- script: cd dist && tfx extension publish -t $(patSecret)
  displayName: TFX Publish Extension