- Tubelator AI
- >
- Videos
- >
- Science & Technology
- >
- Part 4- Creating the Continuous Integration Pipeline (Part -2) - CI/CD in Azure Databricks in Tamil
Part 4- Creating the Continuous Integration Pipeline (Part -2) - CI/CD in Azure Databricks in Tamil
#databricks #azuredatabricks #azuredataengineer #azureintamil #azuretutorialforbeginners #databricksintamil #cicd #azuredevops #devops
In this Part 4 video of CI/CD in Azure Databricks, I have discussed about the process of the Continuous Integration (CI) pipeline and explained the different YAML code used for creating the pipeline.
The next parts will be uploaded soon, stay tuned!
Part #5. Creating CI Pipeline (Part 3)
Part #6. Creating CD Pipeline and End to End Testing
Chapters
TimeStamp:
0:00:00 - Intro
0:01:46 - Variable Group
0:05:05 - Pool-VM Image (Compute)
0:13:25 - Environments
0:16:38 - Service Connection
0:23:57 - Parameter Template
0:27:15 - Generating Databricks Token
0:32:10 - Outro
Please like, share and subscribe if you like the content and leave your comments below.
For contact,
Email: [email protected]
Instagram: mrk_talkstech_tamil
#AzureDatabricks #ApacheSpark #Sparkcompute #clusters #notebooks #magiccommands #machinelearning #ETL #CICD
Install Tubelator On Chrome
Video Summary & Chapters
No chapters for this video generated yet.
Video Transcript
Now we can understand the code created for CICD pipeline.
I will tell you the flow of this code.
This is the main file of CICD pipelines.yaml
So what I mean by master file is, we create a pipeline in ASHUT devops
and we create that pipeline using this file
So this is like a master file
So we call this file
we call this deploy notebooks.yaml
this file is called data breaks token.ps1
so in total we have 3 files
one is master file
another is template file
and the third is script file
so we start from master file
so the important thing is
this first 2 lines
you can understand this easily
Trigger is an option
Trigger on main
If there is any change in main branch
This file will be triggered
If you update a change in main branch
This file will be called
This is the trigger option
Trigger off main
This is the syntax of yaml
Yaml file is in json structure
This is used to create CACD pipeline in most of the companies
In ASHU DevOps, we use YAML file to create CACD pipeline
I think you understood what is Trigger
Next is Variables
This is a very interesting thing
We have to create a variable group in ASHU DevOps
We use parameters in this
For example, the name of the dev workspace will be different, the name of the dev resource group will be different, the name of the uat resource group will be different.
So we create a variable group and within that variable group we create the name of the dev environment.
Similarly, we create another variable group for the production environment.
Before explaining the code below, let's see how to create a variable group
I am going to go to ASHU DevOps
I am in ASHU DevOps
In this, there is an option called Pipelines in the left side
I am clicking on this
In Pipelines, there is an option called Library
you can create a variable group in the library
so in the library there is a option called variable groups
so we can create the variable groups by using this
so there is a button called variable group
so we can click this
after clicking the name of the variable group is heard
so we can copy and paste the name of the variable group
