Debugging threads with the Parallel Stacks window

the visual studio team has been doing
some amazing work with the parallel
Stacks window today we're going to see
how to use it to see what's going on
with threads in an application
[Music]
hi welcome to visual studio toolbox I'm
your host Robert Greene and today we are
going to talk about debugging
multi-threaded applications using the
very cool parallel Stacks window and to
do that joining me are Mark and ROM hey
guys
I know
thanks for coming on the show to show us
this stuff
um give us a uh idea if you will about
quickly why this is an important feature
visual studio and then we'll see how we
use it what's the problem we're solving
here sure um I think a lot of the things
that we as developers do we tend to kind
of debug uh at the individual line level
so we'll stop hit a break point and
we'll debug a particular line of code
but there are a class of problems for
multi-threaded applications that kind of
sometimes require you to take a step
back and see the bigger picture of what
your app is doing that it might include
interactions between threads
um and so you may have to marshal or
kind of negotiate how exactly you're
going to access particular resources
when you've got multiple threads and
then so at that point you have to
understand the relationship between
threads and that's where I think the
tool like parallel Stacks the one that
rum's been working on kind of helps you
take a step back and see the bigger
picture there's a multi-threaded up
something that I
set out to write or are my apps
multi-threaded and I might not even know
that
so so so when it comes to threading
usually you're very explicit about
creating a new thread but we also kind
of use asynchronous programming as well
that's kind of more task based that
allows you to kind of do some of the
sophisticated type of parallel
programming and without necessarily
knowing all the details of how threading
works so
um the parallel Stacks window actually
allows you to see views of both threads
and tasks as well that sounds exciting
all right how's it work cool well I've
got this really cool
um um demo that I think will be useful
for us to kind of talk about this
conversation all up and this is actually
an open source project uh my good friend
Christopher Nassar and his colleague
Kevin ghost wrote this application
actually years ago to talk about uh
parallel Stacks window which is a window
we we have
um here and this particular app is a
fairly straightforward application and
we have three cats that are buying and
selling
um cat paraphernalia things like catnip
or tuners all that kind of stuff and as
you can see it's an app here and they
are kind of trying to replicate a kind
of stock market where they're selling
and they're putting these orders and
then we have a transaction history of
the orders being sent through so I'm
going to go ahead and start the
application
and you see Garfield started to sell a
bunch of boxes and tuna and you see the
transactions are starting to fill
everything's going perfectly and
swimmingly but what we imagine as a
developer I'm thinking if I've got three
cats I probably need three threads to
process it and so all of them are
selling and making these orders and
hopefully the transaction listen as you
can see for the last few seconds it's
actually stopped selling and that
shouldn't have happened
um we we actually should have had
continual selling this whole time I'm
talking but as you can see it's stopped
and what we can do with Visual Studio is
help you in a multi-threaded scenario
and that's where I think the power of
parallel Stacks really shines let me
kind of show you what I mean so I'm I'm
gonna pause the debugger and now you can
do that over here by hovering over the
course it's a control alt break if you
want to know the shortcut
and I'm gonna pause the debugger and now
I'm actually stopped right so
traditionally when I want to know about
the threads there's a drop down right
here
that I can list all the threads
associated with the application as you
can see there's quite a big list now I
could literally go into each of these
threads you can see Cat Felix is a
thread on
2720744 right here
I could go into each of those but I'm
not going to do that because that would
take too much time that'd be a better
way yeah exactly this is where I think
the parallel parallel Stacks comes in
um probably what most developers would
do at this point is probably call it the
call stack
um and actually see some stuff really
cool stuff that Ram's been doing already
um you can see
um it's actually in this course deck
it's referencing that it's waiting on a
lock owned by a different thread thread
three one six four eight
and that's not usually a bad thing but
the but the line above it is actually a
known bad situation in this cold stone
so my code is said line 65 we're going
here line 145 and it's waiting right
here right so this is essentially
waiting but this deadlock is saying I
can't go any further in fact the
deadlock is
it's never going to recover ever it's
going to sit here in this deadlock and
so that paused kind of transactions you
saw from Garfield and and so on and so
forth and never going to recover and so
it's important for us to kind of debug
what this is what's going on here
so there is a new way to get to the
parallel Stacks window that I really
want you all to see here it's the view
all threads here and you can get to that
by hitting Ctrl shift and DS so I'm
going to click that and my
and my parallel Stacks window is open up
here leading this out so you can get a
good view this is the parallel Stacks
window down the bottom right here
depending on how many threads you have
you will see the kind of really big
picture remember in some scenarios in
production you can see hundreds or even
thousands of these threads and so using
this drop down is not going to be
practical here but this gives you a
visual representative representation of
your threads and I'm going to explain
just a part of that right here so on
this left side this you see this blue
outline that's actually me
um showing you
um that there's 45 threads involved
along here and it starts off with clear
expired order
and it goes on to this left hand side
and says 44 of threads are waiting on
thread three one six four eight
we are in a deadlock situation with 44
of our threads and never going to
recover and so now I I know now that
it's not just a single thread that's
involved in this deadlock parallel
Stacks is inviting me to look at the
entire
um the entire process and all the
threads running underneath this process
so my job as a developer now is to
understand why I have a deadlock but I
don't necessarily I wouldn't be able to
understand that if I didn't kind of have
this big picture of you I hope that
makes sense that's cool now you say this
is a new feature it's new in 2022 2022
17 5.
so we've actually Rob's been doing some
fantastic work we've had this feature
parallel stacks for a while um but we've
been perfecting it really over the last
uh couple of releases 17
um this latest preview that's right now
that's out right now has all has has
most of the features we're showing today
this is actually we're actually giving
you Insider look today because this is
actually an internal build you can kind
of tell by the top right side we're
showing you all the latest bits um we're
excited about them we want people to be
aware of them as quickly as possible
okay cool
so we have a deadlock but I still am no
closer to figuring out how exactly
there's a problem but where why exactly
exactly how do I get a bit closer to
this right
um I'm going to show you here how I
would approach this problem one of the
ways I would approach this problem
um
just just like taking a step back also
um you know so for you know just to
maybe clarify what folks may actually be
looking at uh so these are actually the
call Stacks of all the threads that are
in the process right we take all the the
call Stacks of each thread and we kind
of start grouping them together and
that's the view that you're seeing right
here
right so
um each node is kind of a fragment of a
call stack like whatever is common we
start smooshing it together and so
that's a kind of the view that we see
and if you can see that there's a kind
of a blue outline on the uh on a couple
nodes on the left and that means that
that's those notes contain the current
call Stacks for example if you go to the
debug location toolbar on the top it
says that the current thread is three
one one four zero right and uh similarly
now if you would hover over the nodes
like 45 threads and 44 threads that
means our current thread 31140 is one of
the 45 threads
uh yeah if you scroll down yeah you
should see there that right there yeah
it's one of those and it's one of the 44
threads above so this is just kind of a
a bird's eye view of you know everything
that's going on in your process so yeah
I just want to uh
you know yeah cool thank you yeah go
ahead that's a perfect Point there's 45
threads here that have the exact same
call stack they're doing the exact same
thing for example if I were if to If
This Were to a web app for example and
maybe I have thousands of users coming
in
um making an order at my site I may see
similar things right I may see thousands
of these threads we kind of categorize
those threads really conveniently for
you to look at that way you don't have
to as a developer go to each switch
between each thread try to figure out
where where it sits in the context of
your app parallel Stacks is the big
picture view you want to start looking
at these types of problems these
multi-threaded problems with and then
you kind of drill down and that's kind
of The Next Step let's kind of drill
down into this problem a bit more here
and you can also just compare the call
stack window and the parallel Stacks
window like it's pretty much uh you know
you can just see that they are it's the
same thing
so let's raise this up a little bit
and this is the call stack of the thread
that's currently the blue outline
okay
and that represents you can see line 65
line 145 that's the same as the bottom
node
and it's exactly the same thing so it's
showing me this is showing me a specific
call stack and this is showing the
parallel Stacks window is showing me
where this call Stacks sits within the
entire process and its relationship to
other Stacks as well
all right and then from there we can go
to the lines of code that are causing
the problem right exactly that yep so so
let's look at what it's saying this is
waiting on a lock
owned by thread three one six four right
we can actually click to that double
clicking on this
and it's saying we have a waiting on a
lock owned by thread three four five
eight first let's think about what a
lock is right so let's say for example I
am I have three threads running in my
application and their three thirds need
to update a file
first I need to look up all three
threads can't access that file at the
same time right so what we need to do is
a locking mechanism allows you to make
sure that all three threads wait their
turn only the only the thread that owns
the lock can do the writing right so
everybody else is waiting so that call
stack is basically letting you know
every time you see a weight it's
somebody waiting their turn
so all these threads awaiting their turn
behind this deadlock which we know now
will never recover right so a deadlock
is basically where thread a is waiting
on a resource that's thread B owns
simultaneously thread B is waking
waiting on a resource that's owned by
thread a they will never get out of this
situation so this app will never get
these 44 threads back ever right so so
now I now now I know
um if I
look at this I need to to kind of get to
the code
and here on the left hand side I've got
my code alongside my parallel Stacks
window
and you can see here I've got a lock on
orders
and then a lock on sellers
right and my job is to kind of find the
other side of that lock is to find the
other side of this Deadlock
uh so three one six four eight is
waiting on three four five eight so the
other side I am locking seller
a blocked seller and I'm waiting for the
lock on order if you if if you can kind
of rethink that through
um both sides are waiting on a lock that
the other side owns this is a
traditional deadlock you will never
recover you have to restart your
application and what we've seen with the
code now is basically I need the locking
order to be in the same order on both
sides so on one side seller then order
on the other side I'm doing order then
seller that Cross Match
allows me to get in this deadlock
situation I need to change the order of
my locking does that make sense
so so from a code perspective from a
workflow perspective of trying to figure
out these kinds of problems we start
from the big picture we start from the
what are the threads doing are they
deadlocked are they busy are they
waiting what are they waiting on
Parallel Stacks get you that big picture
of you right then when you get that
picture view you find the interesting
threads that you want to track down
right so I found out that this thread is
deadlocked with this one
and I switch between them I get to see
actually lines of code my like kind of
last step is lines of code a great
support engineer a few months back
showed me that kind of principle we
start with the big picture whether it's
memory whether it's threading and then
you kind of Whittle your way back down
to code to kind of figure out what you
want how you want to fix it and this is
the kind of situation where you could
have stared at this code for hours not
understanding not really knowing that
the problem is a specific deadlock kind
of thing that you might eventually the
next day call me over and I'd say hey
these aren't in the same order
all the time now you literally see that
there's a deadlock which now focuses you
on looking at the locks why are the lot
why are we deadlocked and then that
could save you hours yeah yeah exactly
we're thinking about the ways in which
we can kind of help you make you realize
what's going on like one of the other
things here just to kind of bring your
attention again the X's the
informational waiting on locks are might
actually be completely benign right you
may be waiting on it but just waiting
your turn
um so that might actually just be a
normal situation but the deadlock kind
of is telling you hey this you're not
just waiting you're waiting gonna be
waiting forever right so this is a yeah
these indicators are helpful well that
is really really cool I'm I'm totally
enjoying all the enhancements to the
parallel Stacks window thanks for
showing us this
can I just show a couple more things
here just to before we round out sure
okay here on the parallel Stacks window
okay
um you actually now I'm focused on my
code and that's usually most of the case
when I'm debugging but you actually can
switch to show external code meaning
this is going to show framework code as
well oh and to show everything in
context with the framework code and how
it got to this particular position
um it's showing you you know some of the
thread pool thread how that kind of gets
instantiated and where these threads are
going
um I can also have a
toggle that shows me the method view so
again I can kind of focus on the methods
that are called as part of my thread
rather than think about this at every
single frame I can think about this as a
method View and so you can see that I'm
I'm in clear expired orders right here I
really think that's a helpful thing yeah
that's cool also if you go back to show
external here I can select
um I can select nodes that I might want
to
um that I might want to share and I can
copy that information out into a notepad
and share which kind of call Stacks the
call Stacks themselves are useful as
part of my debugging so I might want to
copy and share the call Stacks to a note
file maybe I'm creating a bug and I want
to copy the data of the call Stacks I
can literally copy that out copy that
into a notepad and kind of share that
and finally I also have the opportunity
to savor the diagram of my course that's
working literally
save a a file in the PHD format and that
kind of shows literally a image of my
entire session here that's useful
usually when I'm in a part of a session
with my team and I'm trying to share and
communicate details about the Peril
Stacks uh Ramen I missed any of any
other features that we want to share
here before we move on yeah I know I
think these are kind of the main things
the method view is quite helpful in
showing a kind of a caller quality
relationship you know if you want a uh
typically the parallel Stacks might have
a lot of stacks and you want to kind of
focus on you know your method which is
interesting to you you can select that
talking to the method view which will
show the entire Stacks from the point of
view of that method that is selected so
it's quite helpful in uh you know kind
of trimming the stack to a subset of
frames that you're interested in and
yeah
you can also uh kind of Select an entire
stack by just double clicking a node uh
like you can double click uh in the
regular view uh you can just double
click
uh Mark if you could probably just bring
that up um
yeah so uh so you can you can get out of
the method via so you can just double
click one of the nodes on the top the
node headers on the top and that will
select the entire stack and then you can
control C and uh you know paste it to an
email or in some kind of chat so it just
makes for easy way of selecting an
entire stack
yeah hey very cool
so this has been awesome thank you so
much now I did see that if you click the
in view it says threads now but there
was a second choice in there right
yeah
let's cover that in our next episode
what do you think yep sounds great
so hope you guys enjoyed that stick
around for the next episode where we'll
look at tasks we'll see you next time on
Visual Studio toolbox
foreign
[Music]