What’s Cooking for Rubberduck 2.5.x

If you’ve been following the project all along, this isn’t going to be news, but we kind of missed the v2.4.2 milestone we were slated to release back in April, and here we are with our [next] branch (“pre-release” builds) being a whopping 580+ commits ahead of [master] (“green-release” builds). These commits change a lot of things… so much that v2.4.1 will end up being the only “green-release” of the 2.4.x release cycle, and we’ve decided next release will have to be 2.5.0 – but what is it specifically that warrants such delays and the +1 on the minor version number?

ITypeLib

Perhaps the most important set of changes since v1.2 where we introduced an ANTLR-generated parser, this internal API was actually introduced last year, but until relatively recently it was only used to make the unit testing feature fully host-agnostic (i.e. unit testing works in every host application since) and to retrieve project-level precompiler constants, which closed an otherwise desperate gaping hole in Rubberduck’s understanding of the code that’s in the editor. We are also using it to retrieve and manipulate project references, and possibly in other places I don’t recall at the moment.

But this internal API unlocks much more power than that, and until very recently we hadn’t really started tapping into it. During the v2.5.x cycle, we’ll be using it to instantly populate the Code Explorer toolwindow with tree nodes that still drill down to member level – of course Rubberduck won’t know where a procedure is referenced or be able to refactor anything until parsing has actually occurred, but the project should be instantly navigatable regardless.

We have already begun leveraging this ITypeLib API to augment resolver capabilities, notably with regards to member and module attributes: we can now read most of their values without needing to export anything to any temp file.

So what this API does, is that it taps into VBA/VB6’s internal storage: you may not realize, but compiling your VBA code, internally, creates a COM type library. With this API we can safely query this type library and model user code modules and their members just like any other COM type library, e.g. project references. This means Rubberduck is be able to know what interfaces a document module implements – in other words, when we fully leverage this API we will be able to tell that Sheet1 is a Worksheet and that ThisWorkbook is a Workbook… which means a library-specific inspection like “sheet accessed using string” can now work exactly as intended. We already correctly identify event handler procedures in document modules thanks to these new capabilities; it might seem simple on the surface, but knowing that Sheet1 is a Worksheet and that this Worksheet_Change procedure is handling the Change event of that Worksheet interface, requires looking well beyond the code… and a side-effect of this, is that “procedure not used” no longer fires inspection results for them (the inspection already ignored event handler procedures… all it needed was for the resolver to recognize event handlers in document modules as such).

Default Member Resolution

Once again, a tremendous amount of effort went into augmenting resolver capabilities. This piece of the puzzle is the cornerstone that makes everything else fall into place: if we’re able to issue an inspection result when a variable is never referenced, it’s because the resolver processed all the parse trees and located no references to that variable. But it’s much more than just unused variables: the resolver is the literal central nervous system of Rubberduck – if the resolver doesn’t work well, everything else falls apart.

Except, resolving VBA code correctly is hard. We have an inspection that means to flag missing Set keywords, and until recently it would fire false positives whenever implicit default member calls were involved. What’s that? Picture this code:

Range("A1") = Range("B1")

What’s really happening is this:

Global.Range("A1").[_Default] = Global.Range("A1").[_Default]

But in order to know that, Rubberduck needs to know much more about the code than just what the code is saying: it needs to know that Range is an implicitly-qualified member call on Global (or is it? what if that very same code is in the code-behind of the Sheet3 module?), and that it has a default member that’s the target of this assignment on the left-hand side, and the provider of a value on the right-hand side; it needs to know that this default member yields a Variant (and not another object, which may have its own default member, which might yield an object, which may have a default member, which… so yeah, recursive resolution). And once it knows all that, it can warn you about implicit default member assignments, and soon about any implicit default member call – and help you make them explicit!

Bang notation now also resolves correctly. You write this:

Dim rs As ADODB.Recordset
Set rs = conn.Execute(procName)
Debug.Print rs!Field1

Rubberduck sees this:

Dim rs As ADODB.Recordset
Set rs = conn.Execute(procName)
Debug.Print rs.Fields.Item("Field1").Value

…and this means we’ll soon be able to offer quickfixes/refactorings that turn one notation into the other, and vice-versa.

This is where Rubberduck’s resolver is at, and I need to pinch myself to believe just how crazy wicked awesome it’s becoming – it’s not perfect, but I’m positive, and I’ll repeat this even though it’s been the case for a very long while, but no other VBIDE add-in understands VBA as deeply as Rubberduck.

Moq.Mock<T>

Rubberduck uses the Moq framework for its thousands of unit tests. With it, we’re able to inject “mock” implementations of any abstract dependency: unit testing isn’t complete without a mocking framework, and there’s none for VBA, …for now.

The amount of work involved is astounding, but the important and hard parts are working and we’re just a few road-bumps away from having a COM-visible Moq wrapper API that VBA code can consume to mock any class – your Class1 module or ISomething interface, a ListObject Excel table, any Word.Range, ADODB.Connection, or Scripting.FileSystemObject. This is a massive and complete game-changer that takes unit testing VBA code to a whole new level of credibility.


Timeline

To be honest, there isn’t really any timeline on the table: the 2.5.0 green-release will happen when it does. In the meantime you’ll want to keep an eye on pre-release builds: in the next couple of weeks we’ll be polishing the new features, reviewing what few inspection false positives remain, address a number of prioritized bugs (the all-or-nothing collapsing/expanding of grouping grids, for one), and then we’ll be ready.

There’s plenty of work for all levels and skills, you’re welcome to help us!

Introducing the Reference Explorer

Back in the 2.1.x announcement post over a whole year ago, one of the bullet points about the upcoming roadmap said we were going to “make you never want to use the VBE’s Project References dialog ever again“; it took a bit longer than expected, but as far as we can tell, this feature does exactly that.

If you’ve been following the project on social media recently, you already know that the next version of Rubberduck will introduce a very exciting, unique new feature: the Reference Explorer dialog, and the addition of a references node in the Code Explorer tree.

Vanilla-VBE

Since forever, adding a reference to the active project in the VBE is a rather… vanilla experience. Functional, but somewhere between bland and tedious.

What’s wrong with it?

Regardless of what we think of the very 1998-era buttons docked on the side, the dialog works. There’s a list of available libraries (sorted alphabetically), we can browse for unlisted ones, cancel or accept changes, and the libraries that are selected when the dialog is displayed, are conveniently shown at the top of the list!

On a closer look though…

The vanilla-VBE project references dialog
  1. The list of available libraries has the available libraries listed in alphabetical order. You can’t resize the dialog to show more, but you get first-key search. The Scripting runtime’s library name starts with “Microsoft”… which happens to also be the case for a few other libraries; this makes the extremely useful Scripting.Dictionary and Scripting.FileSystemObject classes pretty much hidden until you stumble upon a blog post or a Stack Overflow answer that introduces them.
  2. The selected libraries show up at the top of the list, in priority order. Locked libraries are stacked at the top. You use the up/down arrow buttons to move the selected library up or down, but you can’t move the locked ones.
  3. The priority buttons are used to determine the identifier resolution order; if an identifier exists in two or more libraries, VBA/VB6 binds to the type defined in the library with the highest priority. There’s no visual cue in the list itself to identify the locked-in type libraries, so the Enabled state of these buttons is used to convey that information: you can’t move the locked-in, default references.
  4. The bottom panel is useful… but the path gets cropped if it’s longer than the rather narrow dialog can fit, and you can’t select or copy the text. The actual library version number isn’t shown.

Visual Studio

Let’s take a look at what adding a project reference using the latest version of Microsoft Visual Studio feels like:

The Microsoft Visual Studio 2017 Reference Manager dialog

The dialog can be resized, search is no longer limited to a single character, but still limited to the beginning of the [Name]. The library info is now richer; it moved to the right side, and a panel on the left side determines the contents of the list. Other than that, besides a new [Version] column and a nice dark theme, …the mechanics are pretty much the same as they were 20 years ago: check boxes in a list. Priority is no longer relevant in .NET though – namespaces fixed that.

Rubberduck

This screenshot was taken shortly before the pull request was opened:

The Rubberduck ‘Add/Remove References’ dialog (work in process: release build may differ)
  1. Available libraries appear in a list on the left-hand side of the dialog. Like in Visual Studio, the version number appears next to the library name, and the list is sorted alphabetically. There is no checkbox: instead, the selected library can be moved into the list of referenced libraries.
  2. Referenced libraries appear in a list on the right-hand side of the dialog. Since there is no checkbox, the selected library can be moved back into the list of available libraries.
  3. Priority up/down buttons appear for the selected referenced library, unless it’s locked.
  4. Icons differentiate locked libraries, libraries that were already referenced when the dialog was shown, and libraries that were newly added. In the list of available libraries, recent and pinned libraries have an icon too.
  5. Search works on a “contains” basis, and matches the library name, description, and path. It immediately filters the list of available libraries.
  6. Tabs for quickly accessing type libraries recently referenced, or pinned libraries, or registered. Host-specific project types are in a separate tab, as applicable.
  7. Bottom panel displays the full name and path of the selected type library. The text can be selected and copied into the clipboard.
  8. Browse button allows referencing any project/library that isn’t listed anywhere. If a library can’t be loaded, it will appear in the list as a broken reference, before it’s even tentatively added to the project.

If you haven’t seen it in action yet, here’s a sneak peek:

Of course that’s just the beginning: layout is not completely final, drag-and-drop functionality remains to-do, among other enhancements.

A first iteration of this feature will likely be merged some time next week, and since this is a major, completely new feature, we’ll bump the minor version and that will be Rubberduck 2.4.0, to be released by the end of 2018…

…not too long after the imminent 2.3.1 hotfix release.

If you think this is one of the coolest things a VBE add-in could possibly do, you’re probably not alone. Share the news, and star us on GitHub!