Skip to main content

Posts

MODELING NON-LINGUISTIC CONTEXTUAL SIGNALS IN LSTM LANGUAGE MODELS VIA DOMAIN ADAPTATION

Our latest machine learning paper has been published. link to view MODELING NON-LINGUISTIC CONTEXTUAL SIGNALS IN LSTM LANGUAGE MODELS VIA DOMAIN ADAPTATION Abstract:  When it comes to speech recognition for voice search, it would be advantageous to take into account application information associated with speech queries. However, in practice, the vast majority of queries typically lack such annotations, posing a challenge to train domain-specific language models (LMs). To obtain robust domain LMs, typically a LM which has been pre-trained on general data will be adapted to specific domains. We propose four adaptation schemes to improve the domain performance of long shortterm memory (LSTM) language models, by incorporating application based contextual signals of voice search queries. Most adaptation strategies are shown to be effective, giving up to 21% relative reduction in perplexity relative to a fine-tuned baseline on a heldout domain specific development set. Initial...

Could bitcoin mining lead to exploring internet bandwidth issues?

Highly recommend the original white paper on crypto written by Bitcoin creator Satoshi Nakamoto: https://bitcoin.org/bitcoin.pdf Short read, not too much math, extremely well written and describes in depth not only the motivation and design, but potential vulnerabilities and why they are not practical threats. One interesting problem not addressed is explosive growth in block size. Recall that: - blocks are limited to one every ~10 mins by self adjusting difficulty - blocks must contain all transactions from previous 10 minutes Currently we are doing about 400k transactions per day, and one block = ~1MB. If we assume transaction count is roughly proportional to the amount of people using the currency, we could see 100x or 1000x growth in transaction volume. So we're looking at 100mb or 1GB blocks. This would create a bandwidth issue, every node has to download 1GB every 10 minutes, there would be propagation delays. I don't think this is a huge problem - networks are gettin...
User’s Manual for the Human Brain DRAFT    User’s Manual for the Human Brain DRAFT 1 Introduction 2 How Happiness Works 2 Distinguishing Magic 2 Reading the brain diagram 3 Hormones: Chemical messengers 4 Dopamine the “super hero” hormone 5 Serotonin the “self confidence” hormone 5 Oxytocin the “love” hormone 5 Endorphins the “morphine” hormone 5 Adrenaline (Epinephrine and Norepinephrine) the “anger” hormone 6 There are two brains, not one 6 Testosterone and sex 6 How Happiness Works 7 What we describe as happiness 7 Moments of presence 7 Design flaws with the system 8 Hormone Generator Machines are Dumb 8 Good Hormones Gone Bad 8 Reasoning brain cannot be turned off 9 How Anger Works 9  Introduction When we are born, unfortunately no instruction manual is issued for our most important, complex and powerful tool, our brain. In very recent times, researchers have reverse engineered how the brain works....