How to Compute a Derivative in R

There are many times that you will want to take the derivative of an expression. Taking a derivative is a very common thing to do in Machine Learning. One example is the activation function of a neural network. If you are going to train the neural network using any of the backpropagation techniques, you will need the derivative of the activation function. In this article I will show you how to take the derivative of a function using R.

For this article we will obtain the derivative of the Sigmoid Activation Function. You might be tempted to create a function for the sigmoid. This can be done as follows.

sigmoid <- function(x) {
    1 / ( 1 + exp(-x) )
}

Once the sigmoid function has been created it can easily be queried as follows.

sigmoid(0)

The above code would produce 0.5. Functions are useful when you need to graph the function. However, to take the derivative you must produce an expression. This is done as follows.

library(Ryacas);
x <- Sym("x");
s <- expression(1 / (1+e^-x));
deriv(s,x);

This produces the following.

e^-x * log(e)/(1 + e^-x)^2

If you would like to simplify the above, use the following.

Simplify(deriv(s,x));