If Notepad's anemic text-editing capabilities frustrate you, you might want to try PrimalPad Community Edition, SAPIEN Technologies' free portable text editor. I've found it to be an exceptional tool.
Syntax highlighting is important to me since it dramatically improves the script editing process. Most text editors have weak VBScript coloring support. In contrast, PrimalPad has perfect VBScript support—every single keyword and reserved word is recognized. PrimalPad even includes highlighting for PowerShell and JScript.
Even without syntax highlighting, PrimalPad is still a clear winner over Notepad. PrimalPad numbers lines with a contrasting color to aid navigation. You can also bookmark a specific line and cycle forward or backward through current bookmarks. You don't need multiple windows for multiple documents. If you drag one or more documents onto the PrimalPad window, each will open in a new tab.
PrimalPad is completely portable and comes preconfigured. You can run it from a network, a thumb drive, or even a CD-ROM. If you don't want to leave any trace of PrimalPad on systems where you use it, you can put the PrimalPad.ini file in the same folder as PrimalPad. PrimalPad will then use this .ini file to store settings. If there isn't an .ini file present, PrimalPad will store settings in the user's registry key.
PrimalPad is definitely worth getting—and not just if you need a portable text editor. PrimalPad is a good starter text editor for administrators who work with VBScript and PowerShell scripts. With PrimalPad, there's no learning curve for the UI and the VBScript keyword highlighting is superior to that in high-priced text editors.
One final note: 32-bit and 64-bit versions of PrimalPad are available. If you intend to use PrimalPad as a portable editor, get the 32-bit version. If you intend to use PrimalPad as a basic script editor on x64 Windows and portability isn't a concern, get the x64 version.