CUDA Support in Visual Studio Code with Julia Reid

Hi, my name is Julia, I'm a program manager on 
the Visual C++ team at Microsoft. Today I'm going  
to show you some of the new CUDA development 
support that we have in Visual Studio Code.  
CUDA is a parallel programming platform that 
allows developers to interact directly with  
the GPU, achieving massive amounts of 
parallelism. NVIDIA and Microsoft have  
been partnering together to light up the CUDA 
development experience in Visual Studio Code.  
Last week we announced in the 1.3 release of 
the C++ extension support for CUDA IntelliSense,  
so that's currently available. But you do have 
to have the insiders build of Visual Studio Code  
for that. It will be released in the official 
Visual Studio Code builds later in May. Further  
on the horizon we have Nsight Visual Studio Code 
edition coming. So you might be familiar with  
Nsight Visual Studio edition, or Nsight Eclipse 
edition. Now there is Nsight Studio Code edition  
which will allow you to build and debug CPU and 
GPU code in Visual Studio Code. Today I'll show  
you the newly available CUDA IntelliSense and 
give you a sneak peek of what's to come with  
Nsight Visual Studio Code edition, which will 
be available in the marketplace in the future.
So I just opened Visual Studio Code insiders on my 
Windows Surface Book, but I actually want to do my  
CUDA development on a GPU-optimized VM, so 
I've already spun up an NC-series VM in Azure  
and I'm going to connect to that VM using the 
remote SSH extension in Visual Studio Code,  
which I'll show you how to do in a second here, 
and then once we're connected to that virtual  
machine then I'll show you all of the cool 
new CUDA development support that we have.  
So in order to install the remote SSH extension, 
you can do that in the marketplace. It looks like  
this. I already have it installed, but this is 
where you would install it for the first time if  
you need to do that, and then once you have the 
remote SSH extension installed you'll see this  
remote explorer icon on the left. So you can go 
ahead and click that, and here I'm looking at SSH  
targets. I could also look at remote containers 
or WSL targets since I have those extensions  
installed as well, but we're interested in SSH at 
the moment. And this is a list of SSH targets that  
I've previously connected to with VS Code, but you 
can add a new one by clicking on this plus button.  
So I'm going to connect to this last one in the 
list because that's where my NVIDIA CUDA tool kit  
is installed. That's my NC-series VM and Azure and 
you can see I have my matrix multiplication sample  
CUDA project on it. So I'm just going to hover 
over it and click connect to host in a new window,  
and this is now opening my remote 
window in Visual Studio Code.
It'll prompt me for my password.
There we go, let me go full screen here.
All right, so I can tell that I'm developing in 
my remote environment by checking out the green  
rectangle in the bottom left-hand corner. So here 
I can see the IP address of the remote target that  
I've connected to. So let's open a folder. And 
it's important to understand that the remote  
SSH development experience in Visual Studio Code 
works differently than it does in Visual Studio,  
so nothing is copied over from my local machine, 
everything lives on the remote target itself.  
So I'm going to be opening a sample 
matrix multiplication project today and  
this matrix mul project comes 
with the NVIDIA CUDA toolkit,  
so if you install the NVIDIA CUDA toolkit 
then you'll also have this matrix mul project  
on your system. So it'll ask me for my 
password one more time, since I'm opening a new
folder
and now it's activating extensions. So I have 
some extensions installed on the remote machine.  
Let's first just check out our environment and 
our VS Code setup for this remote SSH target.  
So if I go over to the extensions marketplace 
I'll see which extensions I have installed  
locally. So that's on my surface book and 
then which are installed on the remote target.  
And here's another quick tip: you can just 
click on this cloud icon if you want to install  
the extensions that you have on your local 
machine. If you want to install those on the  
remote machine just click that icon and then 
select the ones that aren't already installed,  
hit ok. That way you don't have to 
search for them again in the marketplace.  
So the first thing I have installed here is the 
C++ extension, so this is what you're going to  
need in order to get that new CUDA IntelliSense 
that I've been talking about, and quick reminder  
that currently the CUDA IntelliSense is available 
with the latest version of the C++ extension plus  
the insider's build of Visual Studio Code, but it 
will be coming to all builds of Visual Studio Code  
later in May. And the other extension I have 
installed here is the Nsight Visual Studio Code  
edition extension. So this is really exciting 
new stuff. Some of you might be familiar with  
Nsight Eclipse edition or Nsight Visual Studio 
edition. So what NVIDIA and Microsoft have  
been partnering together to build is this new 
Nsight Visual Studio Code edition extension,  
and this will bring build and debug support for 
CUDA programs into Visual Studio Code. This is not  
currently available to the public, but 
it will be available in the marketplace,  
so keep an eye out for it and you can 
actually sign up for early interest  
and get updates on when it will be available. 
I'll have a link for that at the end of the demo.
So now if I go to my project and open my 
matrixmul.cu file you'll see that we have  
syntax highlighting and semantic colorization 
for this CUDA file and that's all brand new. So  
you'll also notice that all of the C++ 
IntelliSense features that you have for  
C++ files, you also have those on CUDA files. 
For example, we have things like autocomplete
we have quick info, so if you hover over a 
variable you'll get some information about what  
it is and the type information. We have signature 
help, so if I start typing MatrixMulCUDA, I can  
see here in the completion list some information 
about the signature. If I keep going with that then  
I'll get parameter help, so I know what parameters 
this function accepts and some information about  
what the function is. The C++ extension also 
supports things like find all references, so if  
I right click on a variable and select find all 
references then it'll search across the project.  
I can also rename the variable and you can 
actually do shift enter to preview your  
results before actually making any changes and in 
this refactor preview here you'll see all of your  
confirmed semantic matches automatically checked. 
If there had been unconfirmed matches, things like  
a text match but not necessarily a semantic match, 
so if it's in a comment for example, then those  
would also appear here and you'd have the option 
to select them before confirming the refactor.
All right, so that's a little preview into 
the CUDA IntelliSense that we just released  
but what about the build and debug support? 
So that will be available with the Nsight  
Visual Studio Code edition extension in the 
marketplace soon, but I'll give you a little  
sneak peek as to what that experience will be 
like. So in our project we have a task.json file  
and this is where we define our build tasks. 
So you can see that we have two tasks here  
for this matrix mul: project one is build and 
the other one is rebuild, which triggers a make  
clean before building the project. Now to run these 
tasks all you do is select terminal run build task  
or you can do control shift b because we have 
rebuild marked as the default one. ctrl shift b  
will trigger a rebuild, so in our 
terminal we see that it ran make clean.
And there we go, and now we have our executable 
and our debugging files. So task.json holds our  
build configurations similarly launch.json which 
is also in your project's .vscode folder that  
holds your debug configurations. So if I open that 
right now, I don't have any debug configurations  
defined for this project, but if I click this add 
configuration blue button, then in this menu I have  
the option to add a CUDA debug configuration 
which will invoke CUDA gdb on my Azure VM. So  
I'll select that so all we have to do here is add 
the name of the program, which is just workspace
folder
and we don't have any arguments.
So now if we go back to matrixmul.cu and let's 
set a few break points. Let's first set one  
in our CPU code in our main function, 
and let's set one in the GPU code as well.
Cool, so now if we start the debugger by hitting f5
this will invoke CUDA gdb on the Linux VM.
So we've hit our first breakpoint, and on  
the left here we have our local variables, our 
registers, and we can add variables to watch, we  
can look at our call stack and see which 
breakpoints we have set in matrixmul.cu.  
So remember right now we're just debugging the CPU 
portion of the code and we can step through and  
watch the variables change as we iterate 
through the program and then let's continue  
running, and the next breakpoint we hit 
will be in the GPU portion of our program.
There we go, so now we can see how the registers 
have changed, our local variables have changed  
and in this output in the debug console you can 
see what our focus is. So remember that GPUs are  
highly parallel processors and the specific SM 
warp and lane within warps that you're looking at  
that's called our focus. Now we can actually 
manually change the focus that we are currently  
debugging and you can do that in the status bar 
by selecting this cuda sm12 warp 0 lane 0 button  
and then you can enter a specific SM warp or lane 
numbers. Let's say we do lane one and hit enter.  
So now again in the debug console you 
can see it says it switched our focus.  
This way we can debug specific portions of the 
code regardless of which subset of SM warp or lane  
the issue is on. Another cool feature about 
the VS Code debugger is that you can edit  
breakpoints that they have conditions. So if we 
do when let's say when tx is zero and ty is two.
Then the next time we hit this breakpoint
tx and ty should match the values that we 
specified in the condition. Which they do: tx  
is 0 and ty is 2. You can also use the debug 
console to execute CUDA gdb commands as you  
would when debugging from the terminal. You 
just put a backtick first and then info cuda  
kernels for example and then you'll 
get information about the cuda kernels.
Let's disable our breakpoints.
And continue running the program. 
And the debugger exits successfully.
So this is a sneak peek into the Nsight
Visual Studio Code edition extension  
that will be coming to the marketplace in 
the near future. If you're interested in  
Nsight Visual Studio Code edition and want to get 
updates about when it will become available you  
can sign up with NVIDIA and join their early 
interest list. Thanks for watching this session,  
I hope you're excited about the future of 
CUDA development in Visual Studio Code.  
Be sure to check out all of the live C++
sessions at Pure Virtual C++ on May 3rd.
Thanks!