Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add grid lstm #345

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

add grid lstm #345

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
344c155
add grid lstm
christopher5106 Sep 26, 2016
32bd83e
some corrections
christopher5106 Sep 27, 2016
a89a728
n
christopher5106 Sep 28, 2016
61ae656
initialization of hidden and cell states in the grid module
christopher5106 Sep 28, 2016
97da470
taking lookup table out of module to be more generic
christopher5106 Sep 28, 2016
05c188f
for printing
christopher5106 Sep 28, 2016
6065689
using userPrevCell to initialize cells after forget
christopher5106 Sep 28, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 16 additions & 9 deletions Grid2DLSTM.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ end

function Grid2DLSTM:__init( outputSize, nb_layers, dropout, tie_weights, rho, cell2gate)
parent.__init(self, rho or 9999)
self.inputSize = outputSize
self.inputSize = outputSize -- for compatibility with tostring function in AbstractRecurrent
self.outputSize = outputSize
self.should_tie_weights = tie_weights or true
self.dropout = dropout or 0
Expand Down Expand Up @@ -147,17 +147,24 @@ function Grid2DLSTM:buildModel()
end

function Grid2DLSTM:updateOutput(input)
if (self.step == 1) and not self.cells[0] then
-- the initial state of the cell/hidden states
print("Initializing the cell/hidden states")
self.cells = {[0] = {}}

for L=1,self.nb_layers do
local h_init = torch.zeros(input:size(1), self.outputSize):cuda()
table.insert(self.cells[0], h_init:clone())
table.insert(self.cells[0], h_init:clone()) -- extra initial state for prev_c
if self.step == 1 then
-- the initial state of the cell/hidden states
if self.userPrevCell then
-- print("Initializing the cell/hidden states with user previous cell")
self.cells = { [0] = self.userPrevCell }
else
-- print("Initializing the cell/hidden states with zero")
self.cells = {[0] = {}}

for L=1,self.nb_layers do
local h_init = torch.zeros(input:size(1), self.outputSize):cuda()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You will have to fix this option so it can work without a gpu, i'm getting errors on cpu version

Copy link

@kenkit kenkit Sep 29, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried removing cuda() references but could not get it to work on the rnn-sin demo with 4x2 tensor/table

table.insert(self.cells[0], h_init:clone())
table.insert(self.cells[0], h_init:clone()) -- extra initial state for prev_c
end
end
end

local input_mem_cell = torch.zeros(input:size(1), self.outputSize):float():cuda()

local rnn_inputs = {input_mem_cell, input, unpack(self.cells[self.step-1])}
Expand Down