M06 - Node.js

Introduction

Node.js is an open-source and cross-platform JavaScript runtime environment. Node runs the V8 JavaScript engine outside of the browser - mostly used in the server side programming. It is suitable to write a small demos to real world large server side applications. Node offers its own programming API and it can be expanded with a thousands of 3rd party modules with Node.js’ package ecosystem.

A Node based app is running in a single process without creating a new thread for every request. Node.js provides a set of asynchronous I/O primitives in its standard library that prevent JavaScript code from blocking. For example when Node is performing an I/O operation (like reading from the network, accessing a database or the filesystem) instead of blocking the thread and wasting CPU cycles waiting, Node will resume the operations when the response comes back. This allows Node to handle thousands of concurrent connections with a single server. On the other hand Node is non-blocking, which means it can run many task simultaneously.

You have DOM, document and window objects, when you are coding JavaScript with browser. Remember, those doesn't exist when you are working with Node in the backend. In server side you have (love) all the nice API's what Node is providing through it's modules.

Usually Node is used to create application which is using web protocols like HTTP, TCP, UDP, DNS and SLL or programs which are reading/writing data to the file system or databases.

Links:

Traditional Web Application vs Node.js Event loop

Any other Web Application developed without Node.js typically follows Multi-Threaded Request-Response model. In this model, client sends request to the server then server do some work based on the client's request, prepare to response and send response back to the client. Every request will create a new thread to the server side. If there aren't any thread available to use then client request will wait to previous request finishes. In this way, the web server model is synchronous or said to be a blocking.

Whenever Node.js runs a program, a thread is automatically created at the same time. This thread is the only starting place where your code is going to be executed. Inside of it, the event loop is generated. If the request from the client side is a non-blocking, it will be served immetiately and the response will be sent back to the client. If the request is a blocking one (for example requiring I/O operations), the request will be sent to the worker thread pool. In this scenario, the request have a callback function that will be fired after the request is finished. Worker thread will send request back to the event loop, which send's it back to the client. This way the blocking request can be handled asynchronously in Node.js.

Quote

IO is essentially any action that depends on a resource outside of your application. This means things like databases, external services etc. In addition, NodeJS considers system calls (filesystem access etc.) IO via non-blocking code. Overview of Blocking vs Non-Blocking

Based on the above explanation, we can assume that most server side applications simply deal in IO. They query databases and request external services to provide data to the client application performing the request. This means that most server side applications will spend majority of their execution time waiting on external resources.

Example: use GET request to get all persons from the database

app.get('/persons', async (request, response) => {
  // Make a request to an external database. 
  // While this is happening, server can continue serving other requests.
  const persons = await Person.find({})
  response.json(persons)
})

Thanks to the event loop in Node.js, operations that perform IO can be pushed to the event loop. While we wait for the request to the external resource to finish, we can yield the control of the thread back to the server application, so it may handle new requests.

This is what makes Node.js very powerful:

Simple mental model of operating in a single thread.
Non-blocking IO.

Install Node.js

Node.js can be installed in different ways and platforms. Official packages for all the major platforms are available https://nodejs.org/en/download/. One very convenient way to install Node.js is through a package manager. In this case, every operating system has its own.

Task

Install node to your computer.

When Node.js is installed, you'll have access to the node executable program in the command line. You can check installed node version with -v attribute.

> node -v
v.14.15.1

Editors

While you're free to use whatever editor you wish to use, Visual Studio Code will be used in this course materials.

Testing

Create a one testing.js JavaScript file:

console.log("Just testing!")

Run it with node:

> node testing.js
Just testing!

You should see "Just testing!" printed out to the console where you ran the command above.

Nice

You can run JavaScript files in server side easily with Node.js!

Creating a project

Each Node.js project is actually a package similar to the packages found from Node Package Manager - NPM. Each package is defined by having a package.json file in the root of the project directory. It defines the package information, lists dependencies along their versions and exposes a bunch of scripts that can be ran as build commands.

First create a folder (name it for example test) and go that folder. Give npm init command to create a package.json file.

> mkdir test
> cd test
> npm init

After that you will need to answer a few basic questions and your project initialization package.json file will be generated.

package name: (test) 
version: (1.0.0) 
description: Just a test project
entry point: (index.js) 
test command: 
git repository: 
keywords: 
author: Pasi Manninen
license: (ISC) MIT

Running a project

The usual way to run a Node program is to run the node globally available command and pass the name of the file you want to execute. Use command line and go to your project folder.

Give a below command to run your project and your first Node application should be executed.

> node index.js
Hello Node.js!

Modify your package.json to use npm script to start your project.

{
  "name": "test",
  "version": "0.0.1",
  "description": "First test",
  "main": "index.js",
  "scripts": {
    "start": "node index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Pasi Manninen",
  "license": "ISC"
}

Now you can launch your project with npm start command in command line. This is how commands should be executed to the Node projects.

> npm start

> hello@1.0.0 start /Users/pasi/Node/delete/hello
> node index.js

Hello Node.js!

Read more about package.json from here: What is the file package.json?

Git and Node.js project

Once you've initialized the project, you should create a .gitignore file in your project directory. This will tell git to ignore certain files. Now you want to ignore all the possible installed node_modules.

1	`node_modules`

If you're using a Visual Studio Code you may also want to add the .vscode folder to the list ignored files and folders.

node_modules
.vscode

Optionally you can create any other needed files like readme.md.

Installation complete

Ok, now your Node.js installation should be complete and you can start learning more!

Node package manager

Node Package Manager (npm) is an online repository https://www.npmjs.com/ for the publishing of open-source Node.js projects. This registry is the world's largest Software Registry. The registry contains over 800,000 code packages. npm is free to use and you can download all npm public software packages without any registration or logon. The Node.js community creates useful modules and publishes them as packages in this repository.

Node Package Manager is also a command line tool that installs, updates or uninstalls Node.js packages in your application. npm includes a CLI (Command Line Client) that can be used to download and install software. Modules will be installed your project's node_modules folder.

Examples

Web server returns text

const http = require('http')
const hostname = '127.0.0.1'
const port = 3000

const server = http.createServer((req, res) => {
  res.statusCode = 200
  res.setHeader('Content-Type', 'text/plain')
  res.write('Hello World!')
  res.end()
})

server.listen(port, hostname, () => {
  console.log(`Server running at http://${hostname}:${port}/`)
})

Tip

Above code can be run without creating a project, because it is not using any 3rd party modules (npms).

Web server returns HTML

const http = require('http')
const hostname = '127.0.0.1'
const port = 3000

const server = http.createServer((req, res) => {
  res.statusCode = 200
  res.setHeader('Content-Type', 'text/html')
  res.write('<h1>Hello World!</h1>')
  res.write('<script>alert("Alert")</script>\n')
  res.end()
})

server.listen(port, hostname, () => {
  console.log(`Server running at http://${hostname}:${port}/`)
})

Tip

Above code can be run without creating a project, because it is not using any 3rd party modules (npms).

Web server and filesystem

The fs module enables interacting with the file system from the Node.js application.

To use this module:

const fs = require('fs')

Create input.txt file and store Sample text line in the file. string inside it.

// input.txt
Sample text line in the file.

You can read the file with readFile function. (error, data) defines a callback function which will be called after a file reading has been finished.

fs.readFile('input.txt', (error, data) => {
   if (error) console.error(error)
   else console.log(data.toString())
})

console.log("Program ended")

Above programming will print the following lines to the console. Note how Line 6 will be called before the callback function will be called and line 2 or 3 has been executed.

Program ended
Sample text line in the file.

Note

fs simply provides a wrapper for the standard file operations, like:

fs.readFileSync() to read files synchronously
fs.readFile() to read files asynchronously
fs.watch() for watching changes in files
fs.exists() for checking that file exist
fs.open() for opening the file
fs.stat() for getting file attributes
fs.read() for reading chuck of data from the file
fs.writeFile() for writing to file
fs.close() for closing the file

Command line parameters/arguments

A Node.js application can be given command-line arguments while starting the Node application. Arguments defined at startup are placed in the process.argv table.

Let's test the situation with an example. Let's create an argv.js file with the following content:

console.log(process.argv)

Run it with node:

> node argv.js
[ '/usr/local/bin/node', '/Users/pasi/JamkFullStack/node/argv.js' ]

The output shows that there are two elements in the process.argv table: the program executing the node and the executable file.

Let's try starting the application by adding more arguments after the command:

> node argv.js first second third
[
  '/usr/local/bin/node',
  '/Users/pasi/JamkFullStack/node/argv.js',
  'first',
  'second',
  'third'
]

Tip

Notice, that the added arguments start from position 2 of the progress.argv array.

Let's modify the example a little so that the given arguments are added together and the sum is printed to the console. Help text is printed on the screen, if the correct number of arguments is not given or the arguments are not numbers.

const path = require('path')

if (process.argv.length <= 3 || isNaN(process.argv[2]) || isNaN(process.argv[3])) {
  console.log(`Usage: ${path.basename(__filename)} INT_NUMBER_ONE INT_NUMBER_TWO`)
  process.exit(-1)
}

const first  = process.argv[2]
const second = process.argv[3]
const sum    = parseInt(first) + parseInt(second)
console.log(`Sum is ${sum}.`)

Creating own modules

You can think that Node modules are JavaScript libraries that contain functions, which you can use in your own Node application. Remember that Node contains its own internal modules, which are straightforward to use and you don't need any installations for them. A list of modules can be found here: Node.js Modules.

You need to use require() command to use any 3rd party (or your own) modules.

In the example below, a separate calculator.js file is created as a separate module. The file contains a few functions that are defined with the exports keyword to be used from outside the calculator.js module, i.e. from another JavaScript file that uses the module.

// calculator.js
const sum = (number1, number2) => {
  return number1 + number2
}

const subs = (number1, number2) => {
  return number1 - number2
}

const mult = (number1, number2) => {
  return number1 * number2
}

const div = (number1, number2) => {
  return number1 / number2
}

module.exports = {
  sum, subs, mult, div
}

Above calculator module can be implemented in another JavaScript data as follows:

// demo.js
const calculator = require('./calculator')

let number1 = 10
let number2 = 2

console.log(`${number1} + ${number2} = ${calculator.sum(number1,number2)}`)
console.log(`${number1} - ${number2} = ${calculator.subs(number1,number2)}`)
console.log(`${number1} * ${number2} = ${calculator.mult(number1,number2)}`)
console.log(`${number1} / ${number2} = ${calculator.div(number1,number2)}`)

Will display:

> node demo.js
+ 2 = 12
- 2 = 8
* 2 = 20
/ 2 = 5