Matrix Factorization: Suggested method based on matrix decomposition technique (P2)

Tuesday, 31/12/2019

Tram Ho

Similar to the previous post, after the theory in Part 1 , in this part I will present the algorithm demo. Let’s find out together

1. Building class MF

Initialization function

Input parameters:

Y : Utility matrix, consisting of 3 columns, each column has 3 figures: user_id, item_id, rating.
n_factors : number of hidden dimensions between users and items, default n_factors = 2 .
X : users matrix
W : matrix ratings
lamda : weight the regularization of the loss function to avoid overfitting, default lamda = 0.1
learning_rate : is learning_rate – the weight of Gradient Descent, used to adjust the learning speed., default learning_rate = 2
n_epochs : number of iterations for training, default n_epochs = 50
top : number of suggested items per user. The default is 10 .
filename : File to store evaluation data.

<span class="token keyword">class</span> <span class="token class-name">MF</span> <span class="token punctuation">(</span> <span class="token builtin">object</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span>
    <span class="token keyword">def</span> <span class="token function">__init__</span> <span class="token punctuation">(</span> self <span class="token punctuation">,</span> Y <span class="token punctuation">,</span> n_factors <span class="token operator">=</span> <span class="token number">2</span> <span class="token punctuation">,</span> X <span class="token operator">=</span> <span class="token boolean">None</span> <span class="token punctuation">,</span> W <span class="token operator">=</span> <span class="token boolean">None</span> <span class="token punctuation">,</span> lamda <span class="token operator">=</span> <span class="token number">0.1</span> <span class="token punctuation">,</span> learning_rate <span class="token operator">=</span> <span class="token number">2</span> <span class="token punctuation">,</span> n_epochs <span class="token operator">=</span> <span class="token number">50</span> <span class="token punctuation">,</span> 
                 top <span class="token operator">=</span> <span class="token number">10</span> <span class="token punctuation">,</span> filename <span class="token operator">=</span> <span class="token boolean">None</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        <span class="token keyword">if</span> filename <span class="token punctuation">:</span>
            self <span class="token punctuation">.</span> f <span class="token operator">=</span> <span class="token builtin">open</span> <span class="token punctuation">(</span> filename <span class="token punctuation">,</span> <span class="token string">'a+'</span> <span class="token punctuation">)</span>
        self <span class="token punctuation">.</span> Y <span class="token operator">=</span> Y
        self <span class="token punctuation">.</span> lamda <span class="token operator">=</span> lamda
        self <span class="token punctuation">.</span> n_factors <span class="token operator">=</span> n_factors
        self <span class="token punctuation">.</span> learning_rate <span class="token operator">=</span> learning_rate
        self <span class="token punctuation">.</span> n_epochs <span class="token operator">=</span> n_epochs
        self <span class="token punctuation">.</span> top <span class="token operator">=</span> top
        self <span class="token punctuation">.</span> users_count <span class="token operator">=</span> <span class="token builtin">int</span> <span class="token punctuation">(</span> np <span class="token punctuation">.</span> <span class="token builtin">max</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> Y <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token number">1</span>
        self <span class="token punctuation">.</span> items_count <span class="token operator">=</span> <span class="token builtin">int</span> <span class="token punctuation">(</span> np <span class="token punctuation">.</span> <span class="token builtin">max</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> Y <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> <span class="token number">1</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token number">1</span>
        self <span class="token punctuation">.</span> ratings_count <span class="token operator">=</span> Y <span class="token punctuation">.</span> shape <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span>
        <span class="token keyword">if</span> X <span class="token operator">==</span> <span class="token boolean">None</span> <span class="token punctuation">:</span>
            self <span class="token punctuation">.</span> X <span class="token operator">=</span> np <span class="token punctuation">.</span> random <span class="token punctuation">.</span> randn <span class="token punctuation">(</span> self <span class="token punctuation">.</span> items_count <span class="token punctuation">,</span> n_factors <span class="token punctuation">)</span>
        <span class="token keyword">if</span> W <span class="token operator">==</span> <span class="token boolean">None</span> <span class="token punctuation">:</span>
            self <span class="token punctuation">.</span> W <span class="token operator">=</span> np <span class="token punctuation">.</span> random <span class="token punctuation">.</span> randn <span class="token punctuation">(</span> n_factors <span class="token punctuation">,</span> self <span class="token punctuation">.</span> users_count <span class="token punctuation">)</span>
        self <span class="token punctuation">.</span> Ybar <span class="token operator">=</span> self <span class="token punctuation">.</span> Y <span class="token punctuation">.</span> copy <span class="token punctuation">(</span> <span class="token punctuation">)</span>
        
        self <span class="token punctuation">.</span> bi <span class="token operator">=</span> np <span class="token punctuation">.</span> random <span class="token punctuation">.</span> randn <span class="token punctuation">(</span> self <span class="token punctuation">.</span> items_count <span class="token punctuation">)</span>
        self <span class="token punctuation">.</span> bu <span class="token operator">=</span> np <span class="token punctuation">.</span> random <span class="token punctuation">.</span> randn <span class="token punctuation">(</span> self <span class="token punctuation">.</span> users_count <span class="token punctuation">)</span>
        self <span class="token punctuation">.</span> n_ratings <span class="token operator">=</span> self <span class="token punctuation">.</span> Y <span class="token punctuation">.</span> shape <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span>

class MF ( object ) :

def __init__ ( self , Y , n_factors = 2 , X = None , W = None , lamda = 0.1 , learning_rate = 2 , n_epochs = 50 ,

top = 10 , filename = None ) :

if filename :

self . f = open ( filename , 'a+' )

self . Y = Y

self . lamda = lamda

self . n_factors = n_factors

self . learning_rate = learning_rate

self . n_epochs = n_epochs

self . top = top

self . users_count = int ( np . max ( self . Y [ : , 0 ] ) ) + 1

self . items_count = int ( np . max ( self . Y [ : , 1 ] ) ) + 1

self . ratings_count = Y . shape [ 0 ]

if X == None :

self . X = np . random . randn ( self . items_count , n_factors )

if W == None :

self . W = np . random . randn ( n_factors , self . users_count )

self . Ybar = self . Y . copy ( )

self . bi = np . random . randn ( self . items_count )

self . bu = np . random . randn ( self . users_count )

self . n_ratings = self . Y . shape [ 0 ]

Changing the weights, you can observe the influence of weights on the evaluation results of the algorithm.

GetUserRated () and getItemsRatedByUser ()

The get_user_rated_item(i) returns the list of users who have rated the i item

    <span class="token keyword">def</span> <span class="token function">get_user_rated_item</span> <span class="token punctuation">(</span> self <span class="token punctuation">,</span> i <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        ids <span class="token operator">=</span> np <span class="token punctuation">.</span> where <span class="token punctuation">(</span> i <span class="token operator">==</span> self <span class="token punctuation">.</span> Ybar <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> <span class="token number">1</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">.</span> astype <span class="token punctuation">(</span> <span class="token builtin">int</span> <span class="token punctuation">)</span>
        users <span class="token operator">=</span> self <span class="token punctuation">.</span> Ybar <span class="token punctuation">[</span> ids <span class="token punctuation">,</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">.</span> astype <span class="token punctuation">(</span> <span class="token builtin">int</span> <span class="token punctuation">)</span>
        ratings <span class="token operator">=</span> self <span class="token punctuation">.</span> Ybar <span class="token punctuation">[</span> ids <span class="token punctuation">,</span> <span class="token number">2</span> <span class="token punctuation">]</span>
        
        <span class="token keyword">return</span> <span class="token punctuation">(</span> users <span class="token punctuation">,</span> ratings <span class="token punctuation">)</span>

def get_user_rated_item ( self , i ) :

ids = np . where ( i == self . Ybar [ : , 1 ] ) [ 0 ] . astype ( int )

users = self . Ybar [ ids , 0 ] . astype ( int )

ratings = self . Ybar [ ids , 2 ]

return ( users , ratings )

The get_item_rated_by_user(u) returns a list of items evaluated by the u user

<span class="token keyword">def</span> <span class="token function">get_item_rated_by_user</span> <span class="token punctuation">(</span> self <span class="token punctuation">,</span> u <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        ids <span class="token operator">=</span> np <span class="token punctuation">.</span> where <span class="token punctuation">(</span> u <span class="token operator">==</span> self <span class="token punctuation">.</span> Ybar <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">.</span> astype <span class="token punctuation">(</span> <span class="token builtin">int</span> <span class="token punctuation">)</span>
        items <span class="token operator">=</span> self <span class="token punctuation">.</span> Ybar <span class="token punctuation">[</span> ids <span class="token punctuation">,</span> <span class="token number">1</span> <span class="token punctuation">]</span> <span class="token punctuation">.</span> astype <span class="token punctuation">(</span> <span class="token builtin">int</span> <span class="token punctuation">)</span>
        ratings <span class="token operator">=</span> self <span class="token punctuation">.</span> Ybar <span class="token punctuation">[</span> ids <span class="token punctuation">,</span> <span class="token number">2</span> <span class="token punctuation">]</span>
        
        <span class="token keyword">return</span> <span class="token punctuation">(</span> items <span class="token punctuation">,</span> ratings <span class="token punctuation">)</span>

def get_item_rated_by_user ( self , u ) :

ids = np . where ( u == self . Ybar [ : , 0 ] ) [ 0 ] . astype ( int )

items = self . Ybar [ ids , 1 ] . astype ( int )

return ( items , ratings )

We will use these two functions to optimize the two matrices X and W.

The update X and W functions :

These are the two optimal functions X and W , with the number of loops being fixed at 50 times.

<span class="token keyword">def</span> <span class="token function">updateX</span> <span class="token punctuation">(</span> self <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        <span class="token keyword">for</span> m <span class="token keyword">in</span> <span class="token builtin">range</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> items_count <span class="token punctuation">)</span> <span class="token punctuation">:</span>
            users <span class="token punctuation">,</span> ratings <span class="token operator">=</span> self <span class="token punctuation">.</span> get_user_rated_item <span class="token punctuation">(</span> m <span class="token punctuation">)</span>
            Wm <span class="token operator">=</span> self <span class="token punctuation">.</span> W <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> users <span class="token punctuation">]</span>
            b <span class="token operator">=</span> self <span class="token punctuation">.</span> bu <span class="token punctuation">[</span> users <span class="token punctuation">]</span>
            sum_grad_xm <span class="token operator">=</span> np <span class="token punctuation">.</span> full <span class="token punctuation">(</span> shape <span class="token operator">=</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> X <span class="token punctuation">[</span> m <span class="token punctuation">]</span> <span class="token punctuation">.</span> shape <span class="token punctuation">)</span> <span class="token punctuation">,</span> fill_value <span class="token operator">=</span> <span class="token number">1e</span> <span class="token operator">-</span> <span class="token number">8</span> <span class="token punctuation">)</span>
            sum_grad_bm <span class="token operator">=</span> <span class="token number">1e</span> <span class="token operator">-</span> <span class="token number">8</span>
            <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span> <span class="token punctuation">(</span> <span class="token number">50</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span>
                xm <span class="token operator">=</span> self <span class="token punctuation">.</span> X <span class="token punctuation">[</span> m <span class="token punctuation">]</span>
                error <span class="token operator">=</span> xm <span class="token punctuation">.</span> dot <span class="token punctuation">(</span> Wm <span class="token punctuation">)</span> <span class="token operator">+</span> self <span class="token punctuation">.</span> bi <span class="token punctuation">[</span> m <span class="token punctuation">]</span> <span class="token operator">+</span> b <span class="token operator">-</span> ratings
                grad_xm <span class="token operator">=</span> error <span class="token punctuation">.</span> dot <span class="token punctuation">(</span> Wm <span class="token punctuation">.</span> T <span class="token punctuation">)</span> <span class="token operator">/</span> self <span class="token punctuation">.</span> n_ratings <span class="token operator">+</span> self <span class="token punctuation">.</span> lamda <span class="token operator">*</span> xm
                grad_bm <span class="token operator">=</span> np <span class="token punctuation">.</span> <span class="token builtin">sum</span> <span class="token punctuation">(</span> error <span class="token punctuation">)</span> <span class="token operator">/</span> self <span class="token punctuation">.</span> n_ratings
                sum_grad_xm <span class="token operator">+=</span> grad_xm <span class="token operator">**</span> <span class="token number">2</span>
                sum_grad_bm <span class="token operator">+=</span> grad_bm <span class="token operator">**</span> <span class="token number">2</span>
                <span class="token comment"># gradient descent</span>
                self <span class="token punctuation">.</span> X <span class="token punctuation">[</span> m <span class="token punctuation">]</span> <span class="token operator">-=</span> self <span class="token punctuation">.</span> lr <span class="token operator">*</span> grad_xm <span class="token punctuation">.</span> reshape <span class="token punctuation">(</span> <span class="token operator">-</span> <span class="token number">1</span> <span class="token punctuation">)</span> <span class="token operator">/</span> np <span class="token punctuation">.</span> sqrt <span class="token punctuation">(</span> sum_grad_xm <span class="token punctuation">)</span>
                self <span class="token punctuation">.</span> bi <span class="token punctuation">[</span> m <span class="token punctuation">]</span> <span class="token operator">-=</span> self <span class="token punctuation">.</span> lr <span class="token operator">*</span> grad_bm <span class="token operator">/</span> np <span class="token punctuation">.</span> sqrt <span class="token punctuation">(</span> sum_grad_bm <span class="token punctuation">)</span>
        
    <span class="token keyword">def</span> <span class="token function">updateW</span> <span class="token punctuation">(</span> self <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        <span class="token keyword">for</span> n <span class="token keyword">in</span> <span class="token builtin">range</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> users_count <span class="token punctuation">)</span> <span class="token punctuation">:</span>
            items <span class="token punctuation">,</span> ratings <span class="token operator">=</span> self <span class="token punctuation">.</span> get_item_rated_by_user <span class="token punctuation">(</span> n <span class="token punctuation">)</span>
            Xn <span class="token operator">=</span> self <span class="token punctuation">.</span> X <span class="token punctuation">[</span> items <span class="token punctuation">,</span> <span class="token punctuation">:</span> <span class="token punctuation">]</span>
            b <span class="token operator">=</span> self <span class="token punctuation">.</span> bi <span class="token punctuation">[</span> items <span class="token punctuation">]</span>
            sum_grad_wn <span class="token operator">=</span> np <span class="token punctuation">.</span> full <span class="token punctuation">(</span> shape <span class="token operator">=</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> W <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> n <span class="token punctuation">]</span> <span class="token punctuation">.</span> shape <span class="token punctuation">)</span> <span class="token punctuation">,</span> fill_value <span class="token operator">=</span> <span class="token number">1e</span> <span class="token operator">-</span> <span class="token number">8</span> <span class="token punctuation">)</span> <span class="token punctuation">.</span> T
            sum_grad_bn <span class="token operator">=</span> <span class="token number">1e</span> <span class="token operator">-</span> <span class="token number">8</span>
            <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span> <span class="token punctuation">(</span> <span class="token number">50</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span>
                wn <span class="token operator">=</span> self <span class="token punctuation">.</span> W <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> n <span class="token punctuation">]</span>
                error <span class="token operator">=</span> Xn <span class="token punctuation">.</span> dot <span class="token punctuation">(</span> wn <span class="token punctuation">)</span> <span class="token operator">+</span> self <span class="token punctuation">.</span> bu <span class="token punctuation">[</span> n <span class="token punctuation">]</span> <span class="token operator">+</span> b <span class="token operator">-</span> ratings
                grad_wn <span class="token operator">=</span> Xn <span class="token punctuation">.</span> T <span class="token punctuation">.</span> dot <span class="token punctuation">(</span> error <span class="token punctuation">)</span> <span class="token operator">/</span> self <span class="token punctuation">.</span> n_ratings <span class="token operator">+</span> self <span class="token punctuation">.</span> lamda <span class="token operator">*</span> wn
                grad_bn <span class="token operator">=</span> np <span class="token punctuation">.</span> <span class="token builtin">sum</span> <span class="token punctuation">(</span> error <span class="token punctuation">)</span> <span class="token operator">/</span> self <span class="token punctuation">.</span> n_ratings
                sum_grad_wn <span class="token operator">+=</span> grad_wn <span class="token operator">**</span> <span class="token number">2</span>
                sum_grad_bn <span class="token operator">+=</span> grad_bn <span class="token operator">**</span> <span class="token number">2</span>
                <span class="token comment"># gradient descent</span>
                self <span class="token punctuation">.</span> W <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> n <span class="token punctuation">]</span> <span class="token operator">-=</span> self <span class="token punctuation">.</span> lr <span class="token operator">*</span> grad_wn <span class="token punctuation">.</span> reshape <span class="token punctuation">(</span> <span class="token operator">-</span> <span class="token number">1</span> <span class="token punctuation">)</span> <span class="token operator">/</span> np <span class="token punctuation">.</span> sqrt <span class="token punctuation">(</span> sum_grad_wn <span class="token punctuation">)</span>
                self <span class="token punctuation">.</span> bu <span class="token punctuation">[</span> n <span class="token punctuation">]</span> <span class="token operator">-=</span> self <span class="token punctuation">.</span> lr <span class="token operator">*</span> grad_bn <span class="token operator">/</span> np <span class="token punctuation">.</span> sqrt <span class="token punctuation">(</span> sum_grad_bn <span class="token punctuation">)</span>

def updateX ( self ) :

for m in range ( self . items_count ) :

users , ratings = self . get_user_rated_item ( m )

Wm = self . W [ : , users ]

b = self . bu [ users ]

sum_grad_xm = np . full ( shape = ( self . X [ m ] . shape ) , fill_value = 1e - 8 )

sum_grad_bm = 1e - 8

for i in range ( 50 ) :

xm = self . X [ m ]

error = xm . dot ( Wm ) + self . bi [ m ] + b - ratings

grad_xm = error . dot ( Wm . T ) / self . n_ratings + self . lamda * xm

grad_bm = np . sum ( error ) / self . n_ratings

sum_grad_xm += grad_xm ** 2

sum_grad_bm += grad_bm ** 2

# gradient descent

self . X [ m ] -= self . lr * grad_xm . reshape ( - 1 ) / np . sqrt ( sum_grad_xm )

self . bi [ m ] -= self . lr * grad_bm / np . sqrt ( sum_grad_bm )

def updateW ( self ) :

for n in range ( self . users_count ) :

items , ratings = self . get_item_rated_by_user ( n )

Xn = self . X [ items , : ]

b = self . bi [ items ]

sum_grad_wn = np . full ( shape = ( self . W [ : , n ] . shape ) , fill_value = 1e - 8 ) . T

sum_grad_bn = 1e - 8

wn = self . W [ : , n ]

error = Xn . dot ( wn ) + self . bu [ n ] + b - ratings

grad_wn = Xn . T . dot ( error ) / self . n_ratings + self . lamda * wn

grad_bn = np . sum ( error ) / self . n_ratings

sum_grad_wn += grad_wn ** 2

sum_grad_bn += grad_bn ** 2

# gradient descent

self . W [ : , n ] -= self . lr * grad_wn . reshape ( - 1 ) / np . sqrt ( sum_grad_wn )

self . bu [ n ] -= self . lr * grad_bn / np . sqrt ( sum_grad_bn )

Main algorithm

    <span class="token keyword">def</span> <span class="token function">fit</span> <span class="token punctuation">(</span> self <span class="token punctuation">,</span> x <span class="token punctuation">,</span> data_size <span class="token punctuation">,</span> Data_test <span class="token punctuation">,</span> test_size <span class="token operator">=</span> <span class="token number">0</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> n_epochs <span class="token punctuation">)</span> <span class="token punctuation">:</span>
            self <span class="token punctuation">.</span> updateW <span class="token punctuation">(</span> <span class="token punctuation">)</span>
            self <span class="token punctuation">.</span> updateX <span class="token punctuation">(</span> <span class="token punctuation">)</span>
            <span class="token keyword">if</span> <span class="token punctuation">(</span> i <span class="token operator">+</span> <span class="token number">1</span> <span class="token punctuation">)</span> <span class="token operator">%</span> x <span class="token operator">==</span> <span class="token number">0</span> <span class="token punctuation">:</span>
                self <span class="token punctuation">.</span> RMSE <span class="token punctuation">(</span> Data_test <span class="token punctuation">,</span> data_size <span class="token operator">=</span> data_size <span class="token punctuation">,</span> test_size <span class="token operator">=</span> <span class="token number">0</span> <span class="token punctuation">,</span> p <span class="token operator">=</span> i <span class="token operator">+</span> <span class="token number">1</span> <span class="token punctuation">)</span>
<span class="token comment">#                 self.evaluate(data_size, Data_test, test_size = 0)</span>

    <span class="token keyword">def</span> <span class="token function">pred</span> <span class="token punctuation">(</span> self <span class="token punctuation">,</span> u <span class="token punctuation">,</span> i <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        u <span class="token operator">=</span> <span class="token builtin">int</span> <span class="token punctuation">(</span> u <span class="token punctuation">)</span>
        i <span class="token operator">=</span> <span class="token builtin">int</span> <span class="token punctuation">(</span> i <span class="token punctuation">)</span>
        pred <span class="token operator">=</span> self <span class="token punctuation">.</span> X <span class="token punctuation">[</span> i <span class="token punctuation">,</span> <span class="token punctuation">:</span> <span class="token punctuation">]</span> <span class="token punctuation">.</span> dot <span class="token punctuation">(</span> self <span class="token punctuation">.</span> W <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> u <span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token operator">+</span> self <span class="token punctuation">.</span> bi <span class="token punctuation">[</span> i <span class="token punctuation">]</span> <span class="token operator">+</span> self <span class="token punctuation">.</span> bu <span class="token punctuation">[</span> u <span class="token punctuation">]</span>
        
        <span class="token keyword">return</span> <span class="token builtin">max</span> <span class="token punctuation">(</span> <span class="token number">0</span> <span class="token punctuation">,</span> <span class="token builtin">min</span> <span class="token punctuation">(</span> <span class="token number">5</span> <span class="token punctuation">,</span> pred <span class="token punctuation">)</span> <span class="token punctuation">)</span>
    
    <span class="token keyword">def</span> <span class="token function">recommend</span> <span class="token punctuation">(</span> self <span class="token punctuation">,</span> u <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        ids <span class="token operator">=</span> np <span class="token punctuation">.</span> where <span class="token punctuation">(</span> self <span class="token punctuation">.</span> Y <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token operator">==</span> u <span class="token punctuation">)</span> <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">.</span> astype <span class="token punctuation">(</span> <span class="token builtin">int</span> <span class="token punctuation">)</span>
        items_rated_by_user <span class="token operator">=</span> self <span class="token punctuation">.</span> Y <span class="token punctuation">[</span> ids <span class="token punctuation">,</span> <span class="token number">1</span> <span class="token punctuation">]</span> <span class="token punctuation">.</span> tolist <span class="token punctuation">(</span> <span class="token punctuation">)</span>
        a <span class="token operator">=</span> np <span class="token punctuation">.</span> zeros <span class="token punctuation">(</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> items_count <span class="token punctuation">,</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span>
        recommended_items <span class="token operator">=</span> <span class="token punctuation">[</span> <span class="token punctuation">]</span>
        pred <span class="token operator">=</span> self <span class="token punctuation">.</span> X <span class="token punctuation">.</span> dot <span class="token punctuation">(</span> self <span class="token punctuation">.</span> W <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> u <span class="token punctuation">]</span> <span class="token punctuation">)</span>
        <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> items_count <span class="token punctuation">)</span> <span class="token punctuation">:</span>
            <span class="token keyword">if</span> i <span class="token operator">not</span> <span class="token keyword">in</span> items_rated_by_user <span class="token punctuation">:</span>
                a <span class="token punctuation">[</span> i <span class="token punctuation">]</span> <span class="token operator">=</span> pred <span class="token punctuation">[</span> i <span class="token punctuation">]</span> <span class="token operator">+</span> self <span class="token punctuation">.</span> bi <span class="token punctuation">[</span> i <span class="token punctuation">]</span> <span class="token operator">+</span> self <span class="token punctuation">.</span> bu <span class="token punctuation">[</span> u <span class="token punctuation">]</span>
        <span class="token keyword">if</span> <span class="token builtin">len</span> <span class="token punctuation">(</span> a <span class="token punctuation">)</span> <span class="token operator">&lt;</span> self <span class="token punctuation">.</span> top <span class="token punctuation">:</span>
            recommended_items <span class="token operator">=</span> np <span class="token punctuation">.</span> argsort <span class="token punctuation">(</span> a <span class="token punctuation">)</span> <span class="token punctuation">[</span> <span class="token operator">-</span> self <span class="token punctuation">.</span> items_count <span class="token punctuation">:</span> <span class="token punctuation">]</span>
        <span class="token keyword">else</span> <span class="token punctuation">:</span>
            recommended_items <span class="token operator">=</span> np <span class="token punctuation">.</span> argsort <span class="token punctuation">(</span> a <span class="token punctuation">)</span> <span class="token punctuation">[</span> <span class="token operator">-</span> self <span class="token punctuation">.</span> top <span class="token punctuation">:</span> <span class="token punctuation">]</span>
        recommended_items <span class="token operator">=</span> np <span class="token punctuation">.</span> where <span class="token punctuation">(</span> a <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">]</span> <span class="token operator">&gt;</span> <span class="token number">0</span> <span class="token punctuation">)</span> <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">.</span> astype <span class="token punctuation">(</span> <span class="token builtin">int</span> <span class="token punctuation">)</span>

<span class="token comment">#         return random.sample(list(recommended_items), self.top)</span>
        <span class="token keyword">return</span> recommended_items <span class="token punctuation">[</span> <span class="token punctuation">:</span> self <span class="token punctuation">.</span> limit <span class="token punctuation">]</span>
<span class="token comment">#         return recommended_items</span>

def fit ( self , x , data_size , Data_test , test_size = 0 ) :

for i in range ( self . n_epochs ) :

self . updateW ( )

self . updateX ( )

if ( i + 1 ) % x == 0 :

self . RMSE ( Data_test , data_size = data_size , test_size = 0 , p = i + 1 )

# self.evaluate(data_size, Data_test, test_size = 0)

def pred ( self , u , i ) :

u = int ( u )

i = int ( i )

pred = self . X [ i , : ] . dot ( self . W [ : , u ] ) + self . bi [ i ] + self . bu [ u ]

return max ( 0 , min ( 5 , pred ) )

def recommend ( self , u ) :

ids = np . where ( self . Y [ : , 0 ] == u ) [ 0 ] . astype ( int )

items_rated_by_user = self . Y [ ids , 1 ] . tolist ( )

a = np . zeros ( ( self . items_count , ) )

recommended_items = [ ]

pred = self . X . dot ( self . W [ : , u ] )

for i in range ( self . items_count ) :

if i not in items_rated_by_user :

a [ i ] = pred [ i ] + self . bi [ i ] + self . bu [ u ]

if len ( a ) < self . top :

recommended_items = np . argsort ( a ) [ - self . items_count : ]

else :

recommended_items = np . argsort ( a ) [ - self . top : ]

recommended_items = np . where ( a [ : ] > 0 ) [ 0 ] . astype ( int )

# return random.sample(list(recommended_items), self.top)

return recommended_items [ : self . limit ]

# return recommended_items

2. Evaluation

Similar to the previous two methods, here I use 2 measures, RMSE and PR :

 <span class="token keyword">def</span> <span class="token function">RMSE</span> <span class="token punctuation">(</span> self <span class="token punctuation">,</span> Data_test <span class="token punctuation">,</span> test_size <span class="token operator">=</span> <span class="token number">0</span> <span class="token punctuation">,</span> data_size <span class="token operator">=</span> <span class="token string">'100K'</span> <span class="token punctuation">,</span> p <span class="token operator">=</span> <span class="token number">10</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        n_tests <span class="token operator">=</span> Data_test <span class="token punctuation">.</span> shape <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span>
        SE <span class="token operator">=</span> <span class="token number">0</span>
        <span class="token keyword">for</span> n <span class="token keyword">in</span> <span class="token builtin">range</span> <span class="token punctuation">(</span> n_tests <span class="token punctuation">)</span> <span class="token punctuation">:</span>
            pred <span class="token operator">=</span> self <span class="token punctuation">.</span> pred <span class="token punctuation">(</span> Data_test <span class="token punctuation">[</span> n <span class="token punctuation">,</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">,</span> Data_test <span class="token punctuation">[</span> n <span class="token punctuation">,</span> <span class="token number">1</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span>
            SE <span class="token operator">+=</span> <span class="token punctuation">(</span> pred <span class="token operator">-</span> Data_test <span class="token punctuation">[</span> n <span class="token punctuation">,</span> <span class="token number">2</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span> <span class="token operator">**</span> <span class="token number">2</span> 
        RMSE <span class="token operator">=</span> np <span class="token punctuation">.</span> sqrt <span class="token punctuation">(</span> SE <span class="token operator">/</span> n_tests <span class="token punctuation">)</span>
        
        <span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string">'%s::1::%d::%d::%r::%r::%rrn'</span> <span class="token operator">%</span> <span class="token punctuation">(</span> <span class="token builtin">str</span> <span class="token punctuation">(</span> data_size <span class="token punctuation">)</span> <span class="token punctuation">,</span> self <span class="token punctuation">.</span> n_factors <span class="token punctuation">,</span> self <span class="token punctuation">.</span> n_epochs <span class="token punctuation">,</span> self <span class="token punctuation">.</span> lamda <span class="token punctuation">,</span> self <span class="token punctuation">.</span> lr <span class="token punctuation">,</span> RMSE <span class="token punctuation">)</span> <span class="token punctuation">)</span>
        self <span class="token punctuation">.</span> f <span class="token punctuation">.</span> write <span class="token punctuation">(</span> <span class="token string">'%s::1::%d::%d::%d::%r::%r::%rrn'</span> <span class="token operator">%</span> <span class="token punctuation">(</span> <span class="token builtin">str</span> <span class="token punctuation">(</span> data_size <span class="token punctuation">)</span> <span class="token punctuation">,</span> self <span class="token punctuation">.</span> n_factors <span class="token punctuation">,</span> self <span class="token punctuation">.</span> n_epochs <span class="token punctuation">,</span> p <span class="token punctuation">,</span> self <span class="token punctuation">.</span> lamda <span class="token punctuation">,</span> self <span class="token punctuation">.</span> lr <span class="token punctuation">,</span> RMSE <span class="token punctuation">)</span> <span class="token punctuation">)</span>
        
        <span class="token keyword">return</span> RMSE
    
    <span class="token keyword">def</span> <span class="token function">evaluate</span> <span class="token punctuation">(</span> self <span class="token punctuation">,</span> data_size <span class="token punctuation">,</span> Data_test <span class="token punctuation">,</span> test_size <span class="token operator">=</span> <span class="token number">0</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span>
        sum_p <span class="token operator">=</span> <span class="token number">0</span>
        sum_r <span class="token operator">=</span> <span class="token number">0</span>
        self <span class="token punctuation">.</span> Pu <span class="token operator">=</span> np <span class="token punctuation">.</span> zeros <span class="token punctuation">(</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> users_count <span class="token punctuation">,</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span>
        <span class="token keyword">for</span> u <span class="token keyword">in</span> <span class="token builtin">range</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> users_count <span class="token punctuation">)</span> <span class="token punctuation">:</span>
            recommended_items <span class="token operator">=</span> self <span class="token punctuation">.</span> recommend <span class="token punctuation">(</span> u <span class="token punctuation">)</span>
            ids <span class="token operator">=</span> np <span class="token punctuation">.</span> where <span class="token punctuation">(</span> Data_test <span class="token punctuation">[</span> <span class="token punctuation">:</span> <span class="token punctuation">,</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token operator">==</span> u <span class="token punctuation">)</span> <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span>
            rated_items <span class="token operator">=</span> Data_test <span class="token punctuation">[</span> ids <span class="token punctuation">,</span> <span class="token number">1</span> <span class="token punctuation">]</span>
            <span class="token keyword">for</span> i <span class="token keyword">in</span> recommended_items <span class="token punctuation">:</span>
                <span class="token keyword">if</span> i <span class="token keyword">in</span> rated_items <span class="token punctuation">:</span>
                    self <span class="token punctuation">.</span> Pu <span class="token punctuation">[</span> u <span class="token punctuation">]</span> <span class="token operator">+=</span> <span class="token number">1</span>
            sum_p <span class="token operator">+=</span> self <span class="token punctuation">.</span> Pu <span class="token punctuation">[</span> u <span class="token punctuation">]</span>
        
        p <span class="token operator">=</span> sum_p <span class="token operator">/</span> <span class="token punctuation">(</span> self <span class="token punctuation">.</span> users_count <span class="token operator">*</span> self <span class="token punctuation">.</span> limit <span class="token punctuation">)</span>
        r <span class="token operator">=</span> sum_p <span class="token operator">/</span> <span class="token punctuation">(</span> Data_test <span class="token punctuation">.</span> shape <span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span> <span class="token punctuation">)</span>
        self <span class="token punctuation">.</span> f <span class="token punctuation">.</span> write <span class="token punctuation">(</span> <span class="token string">'%s::1::%d::%d::%d::%r::%r::%rrn'</span> <span class="token operator">%</span> <span class="token punctuation">(</span> <span class="token builtin">str</span> <span class="token punctuation">(</span> data_size <span class="token punctuation">)</span> <span class="token punctuation">,</span> self <span class="token punctuation">.</span> top <span class="token punctuation">,</span> self <span class="token punctuation">.</span> n_factors <span class="token punctuation">,</span> self <span class="token punctuation">.</span> n_epochs <span class="token punctuation">,</span> test_size <span class="token punctuation">,</span> p <span class="token punctuation">,</span> r <span class="token punctuation">)</span> <span class="token punctuation">)</span>
        
        <span class="token keyword">return</span> p <span class="token punctuation">,</span> r

def RMSE ( self , Data_test , test_size = 0 , data_size = '100K' , p = 10 ) :

n_tests = Data_test . shape [ 0 ]

SE = 0

for n in range ( n_tests ) :

pred = self . pred ( Data_test [ n , 0 ] , Data_test [ n , 1 ] )

SE += ( pred - Data_test [ n , 2 ] ) ** 2

RMSE = np . sqrt ( SE / n_tests )

print ( '%s::1::%d::%d::%r::%r::%rrn' % ( str ( data_size ) , self . n_factors , self . n_epochs , self . lamda , self . lr , RMSE ) )

self . f . write ( '%s::1::%d::%d::%d::%r::%r::%rrn' % ( str ( data_size ) , self . n_factors , self . n_epochs , p , self . lamda , self . lr , RMSE ) )

return RMSE

def evaluate ( self , data_size , Data_test , test_size = 0 ) :

sum_p = 0

sum_r = 0

self . Pu = np . zeros ( ( self . users_count , ) )

for u in range ( self . users_count ) :

recommended_items = self . recommend ( u )

ids = np . where ( Data_test [ : , 0 ] == u ) [ 0 ]

rated_items = Data_test [ ids , 1 ]

for i in recommended_items :

if i in rated_items :

self . Pu [ u ] += 1

sum_p += self . Pu [ u ]

p = sum_p / ( self . users_count * self . limit )

r = sum_p / ( Data_test . shape [ 0 ] )

self . f . write ( '%s::1::%d::%d::%d::%r::%r::%rrn' % ( str ( data_size ) , self . top , self . n_factors , self . n_epochs , test_size , p , r ) )

return p , r

3. Demo with Movielen dataset

rs <span class="token operator">=</span> MF <span class="token punctuation">(</span> rate_train <span class="token punctuation">,</span> n_factors <span class="token operator">=</span> <span class="token number">2</span> <span class="token punctuation">,</span> lamda <span class="token operator">=</span> <span class="token number">0.01</span> <span class="token punctuation">,</span> lr <span class="token operator">=</span> <span class="token number">0.1</span> <span class="token punctuation">,</span> n_epochs <span class="token operator">=</span> <span class="token number">20</span> <span class="token punctuation">,</span> filename <span class="token operator">=</span> <span class="token string">'RMSE_100K_MF.dat'</span> <span class="token punctuation">)</span>
rs <span class="token punctuation">.</span> fit <span class="token punctuation">(</span> <span class="token number">10</span> <span class="token punctuation">,</span> <span class="token string">"100K"</span> <span class="token punctuation">,</span> rate_test <span class="token punctuation">)</span>
rs <span class="token punctuation">.</span> f <span class="token punctuation">.</span> close <span class="token punctuation">(</span> <span class="token punctuation">)</span>

rs = MF ( rate_train , n_factors = 2 , lamda = 0.01 , lr = 0.1 , n_epochs = 20 , filename = 'RMSE_100K_MF.dat' )

rs . f . close ( )

The results I obtained are:

100K::1::2::20::0.01::0.1::0.9634817342439627

100K::1::2::20::0.01::0.1::0.9634984986336697

100K::1::2::20::0.01::0.1::0.9634817342439627

100K::1::2::20::0.01::0.1::0.9634984986336697

Change the weights to find the best set of weights

<span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token punctuation">[</span> <span class="token number">50</span> <span class="token punctuation">,</span> <span class="token number">60</span> <span class="token punctuation">]</span> <span class="token punctuation">:</span>
    <span class="token keyword">for</span> j <span class="token keyword">in</span> <span class="token punctuation">[</span> <span class="token number">0.01</span> <span class="token punctuation">,</span> <span class="token number">0.1</span> <span class="token punctuation">,</span> <span class="token number">0.5</span> <span class="token punctuation">,</span> <span class="token number">1</span> <span class="token punctuation">]</span> <span class="token punctuation">:</span>
        <span class="token keyword">for</span> k <span class="token keyword">in</span> <span class="token punctuation">[</span> <span class="token number">0.1</span> <span class="token punctuation">,</span> <span class="token number">0.5</span> <span class="token punctuation">,</span> <span class="token number">0.75</span> <span class="token punctuation">,</span> <span class="token number">1</span> <span class="token punctuation">,</span> <span class="token number">2</span> <span class="token punctuation">]</span> <span class="token punctuation">:</span>
            rs <span class="token operator">=</span> MF <span class="token punctuation">(</span> rate_train <span class="token punctuation">,</span> n_factors <span class="token operator">=</span> i <span class="token punctuation">,</span> lamda <span class="token operator">=</span> <span class="token number">0.1</span> <span class="token punctuation">,</span> lr <span class="token operator">=</span> <span class="token number">0.1</span> <span class="token punctuation">,</span> n_epochs <span class="token operator">=</span> <span class="token number">10</span> <span class="token punctuation">)</span>
            rs <span class="token punctuation">.</span> fit <span class="token punctuation">(</span> <span class="token number">10</span> <span class="token punctuation">,</span> data_size <span class="token operator">=</span> <span class="token string">"1M"</span> <span class="token punctuation">,</span> Data_test <span class="token operator">=</span> rate_test <span class="token punctuation">,</span> test_size <span class="token operator">=</span> <span class="token number">0.1</span> <span class="token punctuation">)</span>
            rs <span class="token punctuation">.</span> f <span class="token punctuation">.</span> close <span class="token punctuation">(</span> <span class="token punctuation">)</span>

for i in [ 50 , 60 ] :

for j in [ 0.01 , 0.1 , 0.5 , 1 ] :

for k in [ 0.1 , 0.5 , 0.75 , 1 , 2 ] :

rs = MF ( rate_train , n_factors = i , lamda = 0.1 , lr = 0.1 , n_epochs = 10 )

rs . fit ( 10 , data_size = "1M" , Data_test = rate_test , test_size = 0.1 )

rs . f . close ( )

Source code and references:

Code

https://machinelearningcoban.com/2017/05/31/matrixfactorization/

Share the news now

Source : Viblo

Matrix Factorization: Suggested method based on matrix decomposition technique (P2)

1. Building class MF

Initialization function

GetUserRated () and getItemsRatedByUser ()

The update X and W functions :

Main algorithm

2. Evaluation

3. Demo with Movielen dataset

TikTok becomes the second largest social platform in South Africa

The fastest depreciating after 9 months of launch, iPhone 14 Pro Max continues to break the bottom in Vietnam

Beginner's guide to R: Introduction

10 essential SublimeText plugins for JavaScript developers