zk/work/ctm-399.md

How To Use Node.js Modules with npm and package.json

The author selected the Open Internet/Free Speech Fund to receive a donation as part of the Write for DOnations program.

Introduction

Because of such features as its speedy Input/Output (I/O) performance and its basis in the well-known JavaScript language, Node.js has quickly become a popular runtime environment for back-end web development. But as interest grows, larger applications are built, and managing the complexity of the codebase and its dependencies becomes more difficult. Node.js organizes this complexity using modules, which are any single JavaScript files containing functions or objects that can be used by other programs or modules. A collection of one or more modules is commonly referred to as a package, and these packages are themselves organized by package managers.

The Node.js Package Manager (npm) is the default and most popular package manager in the Node.js ecosystem, and is primarily used to install and manage external modules in a Node.js project. It is also commonly used to install a wide range of CLI tools and run project scripts. npm tracks the modules installed in a project with the package.json file, which resides in a project's directory and contains:

• All the modules needed for a project and their installed versions
• All the metadata for a project, such as the author, the license, etc.
• Scripts that can be run to automate tasks within the project

As you create more complex Node.js projects, managing your metadata and dependencies with the package.json file will provide you with more predictable builds, since all external dependencies are kept the same. The file will keep track of this information automatically; while you may change the file directly to update your project's metadata, you will seldom need to interact with it directly to manage modules.

In this tutorial, you will manage packages with npm. The first step will be to create and understand the package.json file. You will then use it to keep track of all the modules you install in your project. Finally, you will list your package dependencies, update your packages, uninstall your packages, and perform an audit to find security flaws in your packages.

Prerequisites

To complete this tutorial, you will need:

• Node.js installed on your development machine. This tutorial uses version 18.3.0. To install this on macOS, follow the steps in How to Install Node.js and Create a Local Development Environment on macOS; to install this on Ubuntu 20.04, follow the Installing Using a PPA or Installing using the Node Version Manager section of How To Install Node.js on Ubuntu 20.04. By having Node.js installed you will also have npm installed; this tutorial uses version 8.11.0.

Step 1 — Creating a package.json File

We begin this tutorial by setting up the example project—a fictional Node.js locator module that gets the user's IP address and returns the country of origin. You will not be coding the module in this tutorial. However, the packages you manage would be relevant if you were developing it.

First, you will create a package.json file to store useful metadata about the project and help you manage the project's dependent Node.js modules. As the suffix suggests, this is a JSON (JavaScript Object Notation) file. JSON is a standard format used for sharing, based on JavaScript objects and consisting of data stored as key-value pairs. If you would like to learn more about JSON, read our Introduction to JSON article.

Since a package.json file contains numerous properties, it can be cumbersome to create manually, without copy and pasting a template from somewhere else. To make things easier, npm provides the init command. This is an interactive command that asks you a series of questions and creates a package.json file based on your answers.

Using the init Command

First, set up a project so you can practice managing modules. In your shell, create a new folder called locator:

mkdir <^>locator<^>

Then move into the new folder:

cd <^>locator<^>

Now, initialize the interactive prompt by entering:

npm init

<$>[note] Note: If your code will use Git for version control, create the Git repository first and then run npm init. The command automatically understands that it is in a Git-enabled folder. If a Git remote is set, it automatically fills out the repository, bugs, and homepage fields for your package.json file. If you initialized the repo after creating the package.json file, you will have to add this information in yourself. For more on Git version control, see our Introduction to Git: Installation, Usage, and Branches series. <$>

You will receive the following output:

[secondary_label Output]
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.

See `npm help init` for definitive documentation on these fields
and exactly what they do.

Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.

Press ^C at any time to quit.
package name: (locator)

You will first be prompted for the name of your new project. By default, the command assumes it's the name of the folder you're in. Default values for each property are shown in parentheses (). Since the default value for name will work for this tutorial, press ENTER to accept it.

The next value to enter is version. Along with the name, this field is required if your project will be shared with others in the npm package repository.

<$>[note] Note: Node.js packages are expected to follow the Semantic Versioning (semver) guide. Therefore, the first number will be the MAJOR version number that only changes when the API changes. The second number will be the MINOR version that changes when features are added. The last number will be the PATCH version that changes when bugs are fixed. <$>

Press ENTER so the default version of 1.0.0 is accepted.

The next field is description—a useful string to explain what your Node.js module does. Our fictional locator project would get the user's IP address and return the country of origin. A fitting description would be Finds the country of origin of the incoming request, so type in something like this and press ENTER. The description is very useful when people are searching for your module.

The following prompt will ask you for the entry point. If someone installs and requires your module, what you set in the entry point will be the first part of your program that is loaded. The value needs to be the relative location of a JavaScript file, and will be added to the main property of the package.json. Press ENTER to keep the default value of index.js.

<$>[note] Note: Most modules have an index.js file as the main point of entry. This is the default value for a package.json's main property, which is the point of entry for npm modules. If there is no package.json, Node.js will try to load index.js by default. <$>

Next, you'll be asked for a test command, an executable script or command to run your project tests. In many popular Node.js modules, tests are written and executed with Mocha, Jest, Jasmine, or other test frameworks. Since testing is beyond the scope of this article, leave this option empty for now, and press ENTER to move on.

The init command will then ask for the project's git repository, which may live on a service such as GitHub (for more information, see GitHub's Repository documentation). You won't use this in this example, so leave it empty as well.

After the repository prompt, the command asks for keywords. This property is an array of strings with useful terms that people can use to find your repository. It's best to have a small set of words that are really relevant to your project, so that searching can be more targeted. List these keywords as a string with commas separating each value. For this sample project, type ip,geo,country at the prompt. The finished package.json will have three items in the array for keywords.

The next field in the prompt is author. This is useful for users of your module who want to get in contact with you. For example, if someone discovers an exploit in your module, they can use this to report the problem so that you can fix it. The author field is a string in the following format: "<^>Name<^> \<<^>Email<^>\> (<^>Website<^>)". For example, "Sammy \<sammy@your_domain\> (https://your_domain)" is a valid author. The email and website data are optional—a valid author could just be a name. Add your contact details as an author and confirm with ENTER.

Finally, you'll be prompted for the license. This determines the legal permissions and limitations users will have while using your module. Many Node.js modules are open source, so npm sets the default to ISC.

At this point, you would review your licensing options and decide what's best for your project. For more information on different types of open source licenses, see this license list from the Open Source Initiative. If you do not want to provide a license for a private repository, you can type UNLICENSED at the prompt. For this sample, use the default ISC license, and press ENTER to finish this process.

The init command will now display the package.json file it's going to create. It will look similar to this:

[secondary_label Output]
About to write to /home/<^>sammy<^>/locator/package.json:

{
  "name": "locator",
  "version": "1.0.0",
  "description": "Finds the country of origin of the incoming request",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "ip",
    "geo",
    "country"
  ],
  "author": "<^>Sammy<^> <<^>sammy<^>@<^>your_domain<^>> (https://<^>your_domain<^>)",
  "license": "ISC"
}


Is this OK? (yes)

Once the information matches what you see here, press ENTER to complete this process and create the package.json file. With this file, you can keep a record of modules you install for your project.

Now that you have your package.json file, you can test out installing modules in the next step.

Step 2 — Installing Modules

It is common in software development to use external libraries to perform ancillary tasks in projects. This allows the developer to focus on the business logic and create the application more quickly and efficiently by utilizing tools and code that others have written that accomplish tasks one needs.

For example, if our sample locator module has to make an external API request to get geographical data, we could use an HTTP library to make that task easier. Since our main goal is to return pertinent geographical data to the user, we could install a package that makes HTTP requests easier for us instead of rewriting this code for ourselves, a task that is beyond the scope of our project.

Let's run through this example. In your locator application, you will use the axios library, which will help you make HTTP requests. Install it by entering the following in your shell:

npm install axios --save

You begin this command with npm install, which will install the package (for brevity you can also use npm i). You then list the packages that you want installed, separated by a space. In this case, this is axios. Finally, you end the command with the optional --save parameter, which specifies that axios will be saved as a project dependency.

When the library is installed, you will see output similar to the following:

[secondary_label Output]
...
+ axios@0.27.2
added 5 packages from 8 contributors and audited 5 packages in 0.764s
found 0 vulnerabilities

Now, open the package.json file, using a text editor of your choice. This tutorial will use nano:

nano package.json

You'll see a new property, as highlighted in the following:

[label locator/package.json]
{
  "name": "locator",
  "version": "1.0.0",
  "description": "Finds the country of origin of the incoming request",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "ip",
    "geo",
    "country"
  ],
  "author": "Sammy sammy@your_domain (https://your_domain)",
  "license": "ISC",
  <^>"dependencies": {<^>
    <^>"axios": "^0.27.2"<^>
  <^>}<^>
}

The --save option told npm to update the package.json with the module and version that was just installed. This is great, as other developers working on your projects can easily see what external dependencies are needed.

<$>[note] Note: You may have noticed the ^ before the version number for the axios dependency. Recall that semantic versioning consists of three digits: MAJOR, MINOR, and PATCH. The ^ symbol signifies that any higher MINOR or PATCH version would satisfy this version constraint. If you see ~ at the beginning of a version number, then only higher PATCH versions satisfy the constraint. <$>

When you are finished reviewing package.json, close the file. If you used nano to edit the file, you can do so by pressing CTRL + X and then ENTER.

Development Dependencies

Packages that are used for the development of a project but not for building or running it in production are called development dependencies. They are not necessary for your module or application to work in production, but may be helpful while writing the code.

For example, it's common for developers to use code linters to ensure their code follows best practices and to keep the style consistent. While this is useful for development, this only adds to the size of the distributable without providing a tangible benefit when deployed in production.

Install a linter as a development dependency for your project. Try this out in your shell:

npm i eslint@8.0.0 --save-dev

In this command, you used the --save-dev flag. This will save eslint as a dependency that is only needed for development. Notice also that you added @8.0.0 to your dependency name. When modules are updated, they are tagged with a version. The @ tells npm to look for a specific tag of the module you are installing. Without a specified tag, npm installs the latest tagged version. Open package.json again:

nano package.json

This will show the following:

[label locator/package.json]
{
  "name": "locator",
  "version": "1.0.0",
  "description": "Finds the country of origin of the incoming request",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "ip",
    "geo",
    "country"
  ],
  "author": "Sammy sammy@your_domain (https://your_domain)",
  "license": "ISC",
  "dependencies": {
    "axios": "^0.19.0"
  },
  <^>"devDependencies": {<^>
    <^>"eslint": "^8.0.0"<^>
  <^>}<^>
}

eslint has been saved as a devDependencies, along with the version number you specified earlier. Exit package.json.

Automatically Generated Files: node_modules and package-lock.json

When you first install a package to a Node.js project, npm automatically creates the node_modules folder to store the modules needed for your project and the package-lock.json file that you examined earlier.

Confirm these are in your working directory. In your shell, type ls and press ENTER. You will observe the following output:

[secondary_label Output]
node_modules    package.json    package-lock.json

The node_modules folder contains every installed dependency for your project. In most cases, you should not commit this folder into your version controlled repository. As you install more dependencies, the size of this folder will quickly grow. Furthermore, the package-lock.json file keeps a record of the exact versions installed in a more succinct way, so including node_modules is not necessary.

While the package.json file lists dependencies that tell us the suitable versions that should be installed for the project, the package-lock.json file keeps track of all changes in package.json or node_modules and tells us the exact version of the package installed. You usually commit this to your version controlled repository instead of node_modules, as it's a cleaner representation of all your dependencies.

Installing from package.json

With your package.json and package-lock.json files, you can quickly set up the same project dependencies before you start development on a new project. To demonstrate this, move up a level in your directory tree and create a new folder named cloned_locator in the same directory level as locator:

cd ..
mkdir cloned_locator

Move into your new directory:

cd cloned_locator

Now copy the package.json and package-lock.json files from locator to cloned_locator:

cp ../locator/package.json ../locator/package-lock.json .

To install the required modules for this project, type:

npm i

npm will check for a package-lock.json file to install the modules. If no lock file is available, it would read from the package.json file to determine the installations. It is usually quicker to install from package-lock.json, since the lock file contains the exact version of modules and their dependencies, meaning npm does not have to spend time figuring out a suitable version to install.

When deploying to production, you may want to skip the development dependencies. Recall that development dependencies are stored in the devDependencies section of package.json, and have no impact on the running of your app. When installing modules as part of the deployment process to deploy your application, omit the dev dependencies by running:

npm i --production

The --production flag ignores the devDependencies section during installation. For now, stick with your development build.

Before moving to the next section, return to the locator folder:

cd ../locator

Global Installations

So far, you have been installing npm modules for the locator project. npm also allows you to install packages globally. This means that the package is available to your user in the wider system, like any other shell command. This ability is useful for the many Node.js modules that are CLI tools.

For example, you may want to blog about the locator project that you're currently working on. To do so, you can use a library like Hexo to create and manage your static website blog. Install the Hexo CLI globally like this:

npm i hexo-cli -g

To install a package globally, you append the -g flag to the command.

<$>[note] Note: If you get a permission error trying to install this package globally, your system may require super user privileges to run the command. Try again with sudo npm i hexo-cli -g. <$>

Test that the package was successfully installed by typing:

hexo --version

You will see output similar to:

[secondary_label Output]
hexo-cli: 4.3.0
os: linux 5.15.0-35-generic Ubuntu 22.04 LTS 22.04 LTS (Jammy Jellyfish)
node: 18.3.0
v8: 10.2.154.4-node.8
uv: 1.43.0
zlib: 1.2.11
brotli: 1.0.9
ares: 1.18.1
modules: 108
nghttp2: 1.47.0
napi: 8
llhttp: 6.0.6
openssl: 3.0.3+quic
cldr: 41.0
icu: 71.1
tz: 2022a
unicode: 14.0
ngtcp2: 0.1.0-DEV
nghttp3: 0.1.0-DEV

So far, you have learned how to install modules with npm. You can install packages to a project locally, either as a production or development dependency. You can also install packages based on pre-existing package.json or package-lock.json files, allowing you to develop with the same dependencies as your peers. Finally, you can use the -g flag to install packages globally, so you can access them regardless of whether you're in a Node.js project or not.

Now that you can install modules, in the next section you will practice techniques to administer your dependencies.

Step 3 — Managing Modules

A complete package manager can do a lot more than install modules. npm has over 20 commands relating to dependency management available. In this step, you will:

• List modules you have installed.
• Update modules to a more recent version.
• Uninstall modules you no longer need.
• Perform a security audit on your modules to find and fix security flaws.

While these examples will be done in your locator folder, all of these commands can be run globally by appending the -g flag at the end of them, exactly like you did when installing globally.

Listing Modules

If you would like to know which modules are installed in a project, it would be easier to use the list or ls command instead of reading the package.json directly. To do this, enter:

npm ls

You will see output like this:

[secondary_label Output]
├── axios@0.27.2
└── eslint@8.0.0

The --depth option allows you to specify what level of the dependency tree you want to see. When it's 0, you only see your top level dependencies. If you want to see the entire dependency tree, use the --all argument:

npm ls --all

You will see output like the following:

[secondary_label Output]
├─┬ axios@0.27.2
│ ├── follow-redirects@1.15.1
│ └─┬ form-data@4.0.0
│   ├── asynckit@0.4.0
│   ├─┬ combined-stream@1.0.8
│   │ └── delayed-stream@1.0.0
│   └─┬ mime-types@2.1.35
│     └── mime-db@1.52.0
└─┬ eslint@8.0.0
  ├─┬ @eslint/eslintrc@1.3.0
  │ ├── ajv@6.12.6 deduped
  │ ├── debug@4.3.4 deduped
  │ ├── espree@9.3.2 deduped
  │ ├── globals@13.15.0 deduped
  │ ├── ignore@5.2.0
  │ ├── import-fresh@3.3.0 deduped
  │ ├── js-yaml@4.1.0 deduped
  │ ├── minimatch@3.1.2 deduped
  │ └── strip-json-comments@3.1.1 deduped

. . .

Updating Modules

It is a good practice to keep your npm modules up to date. This improves your likelihood of getting the latest security fixes for a module. Use the outdated command to check if any modules can be updated:

npm outdated

You will get output like the following:

[secondary_label Output]
Package  Current  Wanted  Latest  Location             Depended by
eslint     8.0.0  8.17.0  8.17.0  node_modules/eslint  locator

This command first lists the Package that's installed and the Current version. The Wanted column shows which version satisfies your version requirement in package.json. The Latest column shows the most recent version of the module that was published.

The Location column states where in the dependency tree the package is located. The outdated command has the --depth flag like ls. By default, the depth is 0.

It seems that you can update eslint to a more recent version. Use the update or up command like this:

npm up eslint

The output of the command will contain the version installed:

[secondary_label Output]

removed 7 packages, changed 4 packages, and audited 91 packages in 1s

14 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

To see which version of eslint that you are using now, you can use npm ls using the package name as an argument:

npm ls eslint

The output will resemble the npm ls command you used before, but include only the eslint package's versions:

[secondary_label Output]
└─┬ eslint@8.17.0
  └─┬ eslint-utils@3.0.0
    └── eslint@8.17.0 deduped

If you wanted to update all modules at once, then you would enter:

npm up

Uninstalling Modules

The npm uninstall command can remove modules from your projects. This means the module will no longer be installed in the node_modules folder, nor will it be seen in your package.json and package-lock.json files.

Removing dependencies from a project is a normal activity in the software development lifecycle. A dependency may not solve the problem as advertised, or may not provide a satisfactory development experience. In these cases, it may better to uninstall the dependency and build your own module.

Imagine that axios does not provide the development experience you would have liked for making HTTP requests. Uninstall axios with the uninstall or un command by entering:

npm un axios

Your output will be similar to:

[secondary_label Output]
removed 8 packages, and audited 83 packages in 542ms

13 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

It doesn't explicitly say that axios was removed. To verify that it was uninstalled, list the dependencies once again:

npm ls

Now, we only see that eslint is installed:

[secondary_label Output]
locator@1.0.0 /home/ubuntu/locator
└── eslint@8.17.0

This shows that you have successfully uninstalled the axios package.

Auditing Modules

npm provides an audit command to highlight potential security risks in your dependencies. To see the audit in action, install an outdated version of the request module by running the following:

npm i request@2.60.0

When you install this outdated version of request, you'll notice output similar to the following:

[secondary_label Output]
npm WARN deprecated cryptiles@2.0.5: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated sntp@1.0.9: This module moved to @hapi/sntp. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated boom@2.10.1: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated node-uuid@1.4.8: Use uuid module instead
npm WARN deprecated har-validator@1.8.0: this library is no longer supported
npm WARN deprecated hoek@2.16.3: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated request@2.60.0: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated hawk@3.1.3: This module moved to @hapi/hawk. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.

added 56 packages, and audited 139 packages in 4s

13 packages are looking for funding
  run `npm fund` for details

9 vulnerabilities (5 moderate, 2 high, 2 critical)

To address all issues, run:
  npm audit fix --force

Run `npm audit` for details.

npm is telling you that you have deprecations and vulnerabilities in your dependencies. To get more details, audit your entire project with:

npm audit

The audit command shows tables of output highlighting security flaws:

[secondary_label Output]
# npm audit report

bl  <1.2.3
Severity: moderate
Remote Memory Exposure in bl - https://github.com/advisories/GHSA-pp7h-53gx-mx7r
fix available via `npm audit fix`
node_modules/bl
  request  2.16.0 - 2.86.0
  Depends on vulnerable versions of bl
  Depends on vulnerable versions of hawk
  Depends on vulnerable versions of qs
  Depends on vulnerable versions of tunnel-agent
  node_modules/request

cryptiles  <=4.1.1
Severity: critical
Insufficient Entropy in cryptiles - https://github.com/advisories/GHSA-rq8g-5pc5-wrhr
Depends on vulnerable versions of boom
fix available via `npm audit fix`
node_modules/cryptiles
  hawk  <=9.0.0
  Depends on vulnerable versions of boom
  Depends on vulnerable versions of cryptiles
  Depends on vulnerable versions of hoek
  Depends on vulnerable versions of sntp
  node_modules/hawk

. . .

9 vulnerabilities (5 moderate, 2 high, 2 critical)

To address all issues, run:
  npm audit fix

You can see the path of the vulnerability, and sometimes npm offers ways for you to fix it. You can run the update command as suggested, or you can run the fix subcommand of audit. In your shell, enter:

npm audit fix

You will see similar output to:

[secondary_label Output]
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

added 19 packages, removed 34 packages, changed 13 packages, and audited 124 packages in 3s

14 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

npm was able to safely update two of the packages, decreasing your vulnerabilities by the same amount. However, you still have three deprecations in your dependencies. The audit fix command does not always fix every problem. Although a version of a module may have a security vulnerability, if you update it to a version with a different API then it could break code higher up in the dependency tree.

You can use the --force parameter to ensure the vulnerabilities are gone, like this:

npm audit fix --force

As mentioned before, this is not recommended unless you are sure that it won't break functionality.

Conclusion

In this tutorial, you went through various exercises to demonstrate how Node.js modules are organized into packages, and how these packages are managed by npm. In a Node.js project, you used npm packages as dependencies by creating and maintaining a package.json file—a record of your project's metadata, including what modules you installed. You also used the npm CLI tool to install, update, and remove modules, in addition to listing the dependency tree for your projects and checking and updating modules that are outdated.

In the future, leveraging existing code by using modules will speed up development time, as you don't have to repeat functionality. You will also be able to create your own npm modules, and these will in turn will be managed by others via npm commands. As for next steps, experiment with what you learned in this tutorial by installing and testing the variety of packages out there. See what the ecosystem provides to make problem solving easier. For example, you could try out TypeScript, a superset of JavaScript, or turn your website into mobile apps with Cordova. If you'd like to learn more about Node.js, see our other Node.js tutorials.