HTTPS Digest Autentisiering - Lua HC2

Post Reply
User avatar
gol
Medlem
Posts: 35
Joined: 30 Apr 2014, 08:00
9
Location: Göteborg
Contact:

Publicerar min implementation av HTTPS med Digest-autentisiering till HC2. Koden är i första hand skriven för min speciella tillämpning - anrop av ett REST API mot larmsystemet Vanderbilt SPC - men borde med viss anpassning även kunna fungera för helt andra tillämpningar. (T.ex skulle koden behöva kunna hantera header-parametrar med olika stavningar). Naturligtvis finns det även behov av förbättrad felhantering och säkert även optimering.

Själva Digest-magiken finns i de första hundra raderna. Resten är en open-source-version av md5, skriven i ren Lua-kod, som är medtagen eftersom denna funktion saknas i Fibaros Lua-lib (så vitt jag vet).

Koden fungerar inte att köras som VD utan måste köras som 'scen'.

Hoppas koden kan komma till nytta för någon. I mån av tid försöker jag svara på frågor.

Code: Select all

--[[
%% properties
%% events
%% globals
--]]

--[[
   Digest Request Handler
--]]
local md5 = {}

-- DigestRequest
local function DigestRequest(method, url, data, username, password, callback)
  local http = net.HTTPClient()
  
  function HttpRequest(method, url, data, headers, callback) 
    http:request(url, {
      options = {
        data = data,
        headers = headers,
        method = method,
        timeout = 5000,
        checkCertificate = false,
      },
      success = function(message)
        fibaro:debug('SUCCESS')
        callback(message.data, message.status, message.headers)
      end,
      error = function(err)
        fibaro:debug('ERROR: ' .. err)
      end
    })
  end
  
  function DigestNegotiation(resData, resCode, resHeaders)
    fibaro:debug(resCode)
    if resCode == 401 and resHeaders['WWW-Authenticate'] then  
      local params = {}                                                 
      for k, v in resHeaders['WWW-Authenticate']:gmatch'(%w+)="([^"]*)"' do
        params[k] = v                                                
      end
    
      local nc = "00000001"                                             
      local cnonce = string.format("%08x", os.time())
    
      local ha1 = md5.sumhexa(username .. ':'                      
                              .. params.realm .. ':'                                        
                              .. password) 
      local ha2 = md5.sumhexa(method .. ':'                          
                              .. url)                                                   
      local res = md5.sumhexa(ha1 .. ':'                             
                              .. params.nonce .. ':'                                        
                              .. nc .. ':'                                                  
                              .. cnonce .. ':'                                              
                              .. params.qop .. ':'                                          
                              .. ha2)                                                   
                                                                              
      local digestHeaders = {                                                
        Authorization = 'Digest username="' .. username        
                              .. '", realm="' .. params.realm                                   
                              .. '", nonce="' .. params.nonce                                   
                              .. '", uri="' .. url                                            
                              .. '", cnonce="' .. cnonce                                        
                              .. '", nc='.. nc                                                  
                              .. ', qop="'.. params.qop                                         
                              .. '", response="'.. res ..'"'
      }
      HttpRequest(method, url, data, digestHeaders, callback) 
    end
  end
    
  HttpRequest(method, url, data, {}, DigestNegotiation)
end 

--[[ 
 Examples
 ========

-- Example GET
DigestRequest('GET', 'https://192.168.0.100:8060/endpoint1', '', 'username', 'password', function(resData, resCode, resHeaders)
    fibaro:debug('GET Reply: ' .. resData)
end)
 
-- Example PUT
DigestRequest('PUT', 'https://192.168.0.100:8080/endpoint2', '', 'username', 'password', function(resData, resCode, resHeaders)
  fibaro:debug('PUT Reply: ' .. resData)
end)


--]]

--[[ 
   md5 and bit implementation in pure Lua 
   ------------------------------------------------------------------------

    VERSION     = "md5.lua 1.1.0",
    DESCRIPTION = "MD5 computation in Lua (5.1-3, LuaJIT)",
    URL         = "https://github.com/kikito/md5.lua",

    MIT LICENSE

    Copyright (c) 2013 Enrique García Cota + Adam Baldwin + hanzao + Equi 4 Software

    Permission is hereby granted, free of charge, to any person obtaining a
    copy of this software and associated documentation files (the
    "Software"), to deal in the Software without restriction, including
    without limitation the rights to use, copy, modify, merge, publish,
    distribute, sublicense, and/or sell copies of the Software, and to
    permit persons to whom the Software is furnished to do so, subject to
    the following conditions:

    The above copyright notice and this permission notice shall be included
    in all copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
    OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
    MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
    IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
    CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
    TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
    SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

--]]

-- bit lib implementions

local char, byte, format, rep, sub =
  string.char, string.byte, string.format, string.rep, string.sub
local bit_or, bit_and, bit_not, bit_xor, bit_rshift, bit_lshift

local ok, bit = pcall(require, 'bit')
if ok then
  bit_or, bit_and, bit_not, bit_xor, bit_rshift, bit_lshift = bit.bor, bit.band, bit.bnot, bit.bxor, bit.rshift, bit.lshift
else
  ok, bit = pcall(require, 'bit32')

  if ok then

    bit_not = bit.bnot

    local tobit = function(n)
      return n <= 0x7fffffff and n or -(bit_not(n) + 1)
    end

    local normalize = function(f)
      return function(a,b) return tobit(f(tobit(a), tobit(b))) end
    end

    bit_or, bit_and, bit_xor = normalize(bit.bor), normalize(bit.band), normalize(bit.bxor)
    bit_rshift, bit_lshift = normalize(bit.rshift), normalize(bit.lshift)

  else

    local function tbl2number(tbl)
      local result = 0
      local power = 1
      for i = 1, #tbl do
        result = result + tbl[i] * power
        power = power * 2
      end
      return result
    end

    local function expand(t1, t2)
      local big, small = t1, t2
      if(#big < #small) then
        big, small = small, big
      end
      -- expand small
      for i = #small + 1, #big do
        small[i] = 0
      end
    end

    local to_bits -- needs to be declared before bit_not

    bit_not = function(n)
      local tbl = to_bits(n)
      local size = math.max(#tbl, 32)
      for i = 1, size do
        if(tbl[i] == 1) then
          tbl[i] = 0
        else
          tbl[i] = 1
        end
      end
      return tbl2number(tbl)
    end

    -- defined as local above
    to_bits = function (n)
      if(n < 0) then
        -- negative
        return to_bits(bit_not(math.abs(n)) + 1)
      end
      -- to bits table
      local tbl = {}
      local cnt = 1
      local last
      while n > 0 do
        last      = n % 2
        tbl[cnt]  = last
        n         = (n-last)/2
        cnt       = cnt + 1
      end

      return tbl
    end

    bit_or = function(m, n)
      local tbl_m = to_bits(m)
      local tbl_n = to_bits(n)
      expand(tbl_m, tbl_n)

      local tbl = {}
      for i = 1, #tbl_m do
        if(tbl_m[i]== 0 and tbl_n[i] == 0) then
          tbl[i] = 0
        else
          tbl[i] = 1
        end
      end

      return tbl2number(tbl)
    end

    bit_and = function(m, n)
      local tbl_m = to_bits(m)
      local tbl_n = to_bits(n)
      expand(tbl_m, tbl_n)

      local tbl = {}
      for i = 1, #tbl_m do
        if(tbl_m[i]== 0 or tbl_n[i] == 0) then
          tbl[i] = 0
        else
          tbl[i] = 1
        end
      end

      return tbl2number(tbl)
    end

    bit_xor = function(m, n)
      local tbl_m = to_bits(m)
      local tbl_n = to_bits(n)
      expand(tbl_m, tbl_n)

      local tbl = {}
      for i = 1, #tbl_m do
        if(tbl_m[i] ~= tbl_n[i]) then
          tbl[i] = 1
        else
          tbl[i] = 0
        end
      end

      return tbl2number(tbl)
    end

    bit_rshift = function(n, bits)
      local high_bit = 0
      if(n < 0) then
        -- negative
        n = bit_not(math.abs(n)) + 1
        high_bit = 0x80000000
      end

      local floor = math.floor

      for i=1, bits do
        n = n/2
        n = bit_or(floor(n), high_bit)
      end
      return floor(n)
    end

    bit_lshift = function(n, bits)
      if(n < 0) then
        -- negative
        n = bit_not(math.abs(n)) + 1
      end

      for i=1, bits do
        n = n*2
      end
      return bit_and(n, 0xFFFFFFFF)
    end
  end
end

-- convert little-endian 32-bit int to a 4-char string
local function lei2str(i)
  local f=function (s) return char( bit_and( bit_rshift(i, s), 255)) end
  return f(0)..f(8)..f(16)..f(24)
end

-- convert raw string to big-endian int
local function str2bei(s)
  local v=0
  for i=1, #s do
    v = v * 256 + byte(s, i)
  end
  return v
end

-- convert raw string to little-endian int
local function str2lei(s)
  local v=0
  for i = #s,1,-1 do
    v = v*256 + byte(s, i)
  end
  return v
end

-- cut up a string in little-endian ints of given size
local function cut_le_str(s,...)
  local o, r = 1, {}
  local args = {...}
  for i=1, #args do
    table.insert(r, str2lei(sub(s, o, o + args[i] - 1)))
    o = o + args[i]
  end
  return r
end

local swap = function (w) return str2bei(lei2str(w)) end

-- An MD5 mplementation in Lua, requires bitlib (hacked to use LuaBit from above, ugh)
-- 10/02/2001 jcw@equi4.com

local CONSTS = {
  0xd76aa478, 0xe8c7b756, 0x242070db, 0xc1bdceee,
  0xf57c0faf, 0x4787c62a, 0xa8304613, 0xfd469501,
  0x698098d8, 0x8b44f7af, 0xffff5bb1, 0x895cd7be,
  0x6b901122, 0xfd987193, 0xa679438e, 0x49b40821,
  0xf61e2562, 0xc040b340, 0x265e5a51, 0xe9b6c7aa,
  0xd62f105d, 0x02441453, 0xd8a1e681, 0xe7d3fbc8,
  0x21e1cde6, 0xc33707d6, 0xf4d50d87, 0x455a14ed,
  0xa9e3e905, 0xfcefa3f8, 0x676f02d9, 0x8d2a4c8a,
  0xfffa3942, 0x8771f681, 0x6d9d6122, 0xfde5380c,
  0xa4beea44, 0x4bdecfa9, 0xf6bb4b60, 0xbebfbc70,
  0x289b7ec6, 0xeaa127fa, 0xd4ef3085, 0x04881d05,
  0xd9d4d039, 0xe6db99e5, 0x1fa27cf8, 0xc4ac5665,
  0xf4292244, 0x432aff97, 0xab9423a7, 0xfc93a039,
  0x655b59c3, 0x8f0ccc92, 0xffeff47d, 0x85845dd1,
  0x6fa87e4f, 0xfe2ce6e0, 0xa3014314, 0x4e0811a1,
  0xf7537e82, 0xbd3af235, 0x2ad7d2bb, 0xeb86d391,
  0x67452301, 0xefcdab89, 0x98badcfe, 0x10325476
}

local f=function (x,y,z) return bit_or(bit_and(x,y),bit_and(-x-1,z)) end
local g=function (x,y,z) return bit_or(bit_and(x,z),bit_and(y,-z-1)) end
local h=function (x,y,z) return bit_xor(x,bit_xor(y,z)) end
local i=function (x,y,z) return bit_xor(y,bit_or(x,-z-1)) end
local z=function (ff,a,b,c,d,x,s,ac)
  a=bit_and(a+ff(b,c,d)+x+ac,0xFFFFFFFF)
  -- be *very* careful that left shift does not cause rounding!
  return bit_or(bit_lshift(bit_and(a,bit_rshift(0xFFFFFFFF,s)),s),bit_rshift(a,32-s))+b
end

local function transform(A,B,C,D,X)
  local a,b,c,d=A,B,C,D
  local t=CONSTS

  a=z(f,a,b,c,d,X[ 0], 7,t[ 1])
  d=z(f,d,a,b,c,X[ 1],12,t[ 2])
  c=z(f,c,d,a,b,X[ 2],17,t[ 3])
  b=z(f,b,c,d,a,X[ 3],22,t[ 4])
  a=z(f,a,b,c,d,X[ 4], 7,t[ 5])
  d=z(f,d,a,b,c,X[ 5],12,t[ 6])
  c=z(f,c,d,a,b,X[ 6],17,t[ 7])
  b=z(f,b,c,d,a,X[ 7],22,t[ 8])
  a=z(f,a,b,c,d,X[ 8], 7,t[ 9])
  d=z(f,d,a,b,c,X[ 9],12,t[10])
  c=z(f,c,d,a,b,X[10],17,t[11])
  b=z(f,b,c,d,a,X[11],22,t[12])
  a=z(f,a,b,c,d,X[12], 7,t[13])
  d=z(f,d,a,b,c,X[13],12,t[14])
  c=z(f,c,d,a,b,X[14],17,t[15])
  b=z(f,b,c,d,a,X[15],22,t[16])

  a=z(g,a,b,c,d,X[ 1], 5,t[17])
  d=z(g,d,a,b,c,X[ 6], 9,t[18])
  c=z(g,c,d,a,b,X[11],14,t[19])
  b=z(g,b,c,d,a,X[ 0],20,t[20])
  a=z(g,a,b,c,d,X[ 5], 5,t[21])
  d=z(g,d,a,b,c,X[10], 9,t[22])
  c=z(g,c,d,a,b,X[15],14,t[23])
  b=z(g,b,c,d,a,X[ 4],20,t[24])
  a=z(g,a,b,c,d,X[ 9], 5,t[25])
  d=z(g,d,a,b,c,X[14], 9,t[26])
  c=z(g,c,d,a,b,X[ 3],14,t[27])
  b=z(g,b,c,d,a,X[ 8],20,t[28])
  a=z(g,a,b,c,d,X[13], 5,t[29])
  d=z(g,d,a,b,c,X[ 2], 9,t[30])
  c=z(g,c,d,a,b,X[ 7],14,t[31])
  b=z(g,b,c,d,a,X[12],20,t[32])

  a=z(h,a,b,c,d,X[ 5], 4,t[33])
  d=z(h,d,a,b,c,X[ 8],11,t[34])
  c=z(h,c,d,a,b,X[11],16,t[35])
  b=z(h,b,c,d,a,X[14],23,t[36])
  a=z(h,a,b,c,d,X[ 1], 4,t[37])
  d=z(h,d,a,b,c,X[ 4],11,t[38])
  c=z(h,c,d,a,b,X[ 7],16,t[39])
  b=z(h,b,c,d,a,X[10],23,t[40])
  a=z(h,a,b,c,d,X[13], 4,t[41])
  d=z(h,d,a,b,c,X[ 0],11,t[42])
  c=z(h,c,d,a,b,X[ 3],16,t[43])
  b=z(h,b,c,d,a,X[ 6],23,t[44])
  a=z(h,a,b,c,d,X[ 9], 4,t[45])
  d=z(h,d,a,b,c,X[12],11,t[46])
  c=z(h,c,d,a,b,X[15],16,t[47])
  b=z(h,b,c,d,a,X[ 2],23,t[48])

  a=z(i,a,b,c,d,X[ 0], 6,t[49])
  d=z(i,d,a,b,c,X[ 7],10,t[50])
  c=z(i,c,d,a,b,X[14],15,t[51])
  b=z(i,b,c,d,a,X[ 5],21,t[52])
  a=z(i,a,b,c,d,X[12], 6,t[53])
  d=z(i,d,a,b,c,X[ 3],10,t[54])
  c=z(i,c,d,a,b,X[10],15,t[55])
  b=z(i,b,c,d,a,X[ 1],21,t[56])
  a=z(i,a,b,c,d,X[ 8], 6,t[57])
  d=z(i,d,a,b,c,X[15],10,t[58])
  c=z(i,c,d,a,b,X[ 6],15,t[59])
  b=z(i,b,c,d,a,X[13],21,t[60])
  a=z(i,a,b,c,d,X[ 4], 6,t[61])
  d=z(i,d,a,b,c,X[11],10,t[62])
  c=z(i,c,d,a,b,X[ 2],15,t[63])
  b=z(i,b,c,d,a,X[ 9],21,t[64])

  return bit_and(A+a,0xFFFFFFFF),bit_and(B+b,0xFFFFFFFF),
         bit_and(C+c,0xFFFFFFFF),bit_and(D+d,0xFFFFFFFF)
end

----------------------------------------------------------------

local function md5_update(self, s)
  self.pos = self.pos + #s
  s = self.buf .. s
  for ii = 1, #s - 63, 64 do
    local X = cut_le_str(sub(s,ii,ii+63),4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4)
    assert(#X == 16)
    X[0] = table.remove(X,1) -- zero based!
    self.a,self.b,self.c,self.d = transform(self.a,self.b,self.c,self.d,X)
  end
  self.buf = sub(s, math.floor(#s/64)*64 + 1, #s)
  return self
end

local function md5_finish(self)
  local msgLen = self.pos
  local padLen = 56 - msgLen % 64

  if msgLen % 64 > 56 then padLen = padLen + 64 end

  if padLen == 0 then padLen = 64 end

  local s = char(128) .. rep(char(0),padLen-1) .. lei2str(bit_and(8*msgLen, 0xFFFFFFFF)) .. lei2str(math.floor(msgLen/0x20000000))
  md5_update(self, s)

  assert(self.pos % 64 == 0)
  return lei2str(self.a) .. lei2str(self.b) .. lei2str(self.c) .. lei2str(self.d)
end

----------------------------------------------------------------

function md5.new()
  return { a = CONSTS[65], b = CONSTS[66], c = CONSTS[67], d = CONSTS[68],
           pos = 0,
           buf = '',
           update = md5_update,
           finish = md5_finish }
end

function md5.tohex(s)
  return format("%08x%08x%08x%08x", str2bei(sub(s, 1, 4)), str2bei(sub(s, 5, 8)), str2bei(sub(s, 9, 12)), str2bei(sub(s, 13, 16)))
end

function md5.sum(s)
  return md5.new():update(s):finish()
end

function md5.sumhexa(s)
  return md5.tohex(md5.sum(s))
end
Vanderbilt SPC integration, Z-Wave, Zigbee, MySensors, LoRaWAN, Linux, Raspberry PI, OpenWRT
darylmcclure
Ny medlem
Posts: 4
Joined: 05 Sep 2022, 00:37
1

Firstly apologies for writing in English.

Secondly thanks for posting the LUA code in your link.

I wasn't familiar with Digest authentication till earlier this week and your code was helpful in that process.

That said I haven't managed to get the code to authenticate even using standard url's to check digest auth.

I confirmed via Python request, overriding the underlying method to allow me to input a nonce and cnonce, that your code does return a valid response.

I'm wondering if it's an issue with Fibaro / HC2 and net.Http:request or possibly some subtle formatting / encoding of the headers.

Interested to hear if you managed to get your implementation working on the HC2?
User avatar
gol
Medlem
Posts: 35
Joined: 30 Apr 2014, 08:00
9
Location: Göteborg
Contact:

Yes, the code works on HC2 at least for my specific use case. But the code is not completely generic, for instance you have to adapt it to take care of different ways to spell WWW-Authenticate (that can have any combination of lower/upper case characters).
Vanderbilt SPC integration, Z-Wave, Zigbee, MySensors, LoRaWAN, Linux, Raspberry PI, OpenWRT
darylmcclure
Ny medlem
Posts: 4
Joined: 05 Sep 2022, 00:37
1

thanks for replying

It reads the 'WWW-Authenticate' header ok, e.g. below

[DEBUG] 21:09:45: WWW-Authenticate = Digest qop="auth", realm="DS-0AAC0497", nonce="ZTllNmJmZDNmMjRmMTBmMDYyZGY2YjBlMjFhZmY4MzM=",

it also constructs the header ok too

[DEBUG] 21:09:46: Authorization: Digest username="admin", realm="DS-0AAC0497", nonce="ZTllNmJmZDNmMjRmMTBmMDYyZGY2YjBlMjFhZmY4MzM=", uri="/ISAPI/VideoIntercom/callStatus", qop=auth, nc=00000001, cnonce="f66c07ed373a89b7", response="cba8b608a35f0b25264e468c7992a484", opaque=""

when it makes the second http request it gets a 401 not sure why
darylmcclure
Ny medlem
Posts: 4
Joined: 05 Sep 2022, 00:37
1

I think it may be something peculiar to HIKVISION

Trying a 'test' URL works fine with your code

url = 'https://httpbin.org/digest-auth/auth/user/passwd'
uri = '/digest-auth/auth/user/passwd'
user = 'user'
passwd = 'passwd'

What's curious though is the same http request to HIKVISION doesn't work from the HC2 but works fine using curl or Python requests
darylmcclure
Ny medlem
Posts: 4
Joined: 05 Sep 2022, 00:37
1

I didn't copy the entire WWW-Authenticate header before:

[DEBUG] 21:31:15: WWW-Authenticate = Digest qop="auth", realm="DS-0AAC0497", nonce="Njg5NDFlOTFmYzU0NDlkZTVhYTAyMjg2OWU3OTFjNDc=", stale="false", opaque="", domain="::"

there's nothing special about the parameters:

stale="false", opaque="", domain="::"

that your are aware of?
User avatar
gol
Medlem
Posts: 35
Joined: 30 Apr 2014, 08:00
9
Location: Göteborg
Contact:

Sorry, I don't think I can help you further with this issue. Probably you can find the solution by looking at how Python or curl communicates with your HIKVISION.
Vanderbilt SPC integration, Z-Wave, Zigbee, MySensors, LoRaWAN, Linux, Raspberry PI, OpenWRT
Post Reply