maxmustermann/NeMo
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3936 commits 187 branches 20 tags 354 MiB
Commit graph

3936 commits

This branch
This branch
All branches
Author SHA1 Message Date
Eric Harper 7ce63be27b
Remove PTL 1.4 upper bound (#2600) 
* update exp_manager

Signed-off-by: ericharper <complex451@gmail.com>

* update ptl trainer dataclass

Signed-off-by: ericharper <complex451@gmail.com>

* update check for ranks

Signed-off-by: ericharper <complex451@gmail.com>

* style

Signed-off-by: ericharper <complex451@gmail.com>

* remove returned from val epoch end

Signed-off-by: ericharper <complex451@gmail.com>

* revert

Signed-off-by: ericharper <complex451@gmail.com>

* Try to fix multi validation logging

Signed-off-by: smajumdar <titu1994@gmail.com>

* Try to fix multi validation logging

Signed-off-by: smajumdar <titu1994@gmail.com>

* Try to fix multi validation logging

Signed-off-by: smajumdar <titu1994@gmail.com>

* update for glue with PTL1.4 (#2634)

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* add rank_zero_only to self.log

Signed-off-by: ericharper <complex451@gmail.com>

Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com>
 2021-08-10 12:50:22 -07:00
Yang Zhang 58c78869bd
Tn class eval (#2614) 
* addde comments

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add class level evaluation for duplex text norm

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added comments

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix doc string

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* revert previous change

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix class based eval to work with input with punctuation

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix eval for itn

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add counts

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
 2021-08-10 11:46:43 -07:00
Somshubra Majumdar 4af3986326
Move ASR Webapp (#2632) 
Signed-off-by: smajumdar <titu1994@gmail.com>
 2021-08-10 10:44:49 -06:00
Ryan Leary 2be5853cdb
Add basic grpc MT server (#1807) 
* Add basic grpc MT server

Add readme, server updates

Signed-off-by: Ryan Leary <rleary@nvidia.com>

* style fix

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* fixing license headers

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* Add punctuation model into NMT service

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix merge conflicts

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* style fixes to unblock CI

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* Add a Jarvis ASR + NeMo NMT client

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* style fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Refactor gRPC service

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update license headers

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update one more license header

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Whitepsace in header

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix grpc requirement

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update license headers

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add option to specify src/tgt lang and import fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix unused imports

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Renaming variables

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Co-authored-by: Ryan Leary <rleary@nvidia.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
 2021-08-09 16:31:41 -06:00
Tuan Manh Lai 7e6197d33d
Fixed an error related to the task indicator of the tagger during inference. (#2627) 
* A minor fix to _infer() of DuplexTaggerModel
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Remove tagger data augmentation
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Minor fixes to cache path
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Style fix
Signed-off-by: Tuan Lai <tuanl@nvidia.com>
 2021-08-09 14:32:11 -07:00
Ryan Leary 241855cc2a
Force fastpitch output to fp32 (#2629) 
Signed-off-by: Ryan Leary <rleary@nvidia.com>

Co-authored-by: Ryan Leary <rleary@nvidia.com>
 2021-08-09 12:11:01 -04:00
Tuan Manh Lai 18b51accbd
Evaluate the performance of the decoder for each semiotic class (#2625) 
* Add class_based_decoding_evaluation.py
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Remove unused imports
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Add evaluation for tagger to class_based_eval
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com>
 2021-08-07 16:01:32 -07:00
Tuan Manh Lai 0d41f3a8fe
Allow using covering grammars for neural English TN model (#2602) 
* Compute probability of each sequence
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Use CGs when the model is not confident
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Add script for visualization
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Add comments on how to generate visualizations
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Remove spaces in URLs when using CGs
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Allow setting n_tagged
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Add docstring
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* if there is any exception, fall back to the input
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Minor changes to URL processing
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Style fix
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Add docstrings and comments
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* PYNINI_AVAILABLE check
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com>
 2021-08-07 15:19:57 -07:00
Somshubra Majumdar fa7d95583e
Fix return config path for new SaveRestoreConnector (#2626) 
* Fix return config path for new SaveRestore connector

Signed-off-by: smajumdar <titu1994@gmail.com>

* Fix return config path for new SaveRestore connector

Signed-off-by: smajumdar <titu1994@gmail.com>
 2021-08-06 15:28:24 -07:00
Tuan Manh Lai 2150fbd2c8
Allowed setting the train set size of duplex TN training (#2605) 
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com>
 2021-08-06 12:34:52 -07:00
Eric Harper ac73f9117c
Add save restore connector to ModelPT (#2592) 
* add save restore connector

Signed-off-by: ericharper <complex451@gmail.com>

* add save restore connector

Signed-off-by: ericharper <complex451@gmail.com>

* add save restore connector property

Signed-off-by: ericharper <complex451@gmail.com>

* add _default_save_to

Signed-off-by: ericharper <complex451@gmail.com>

* moving globals to app_state

Signed-off-by: ericharper <complex451@gmail.com>

* moving globals to app_state

Signed-off-by: ericharper <complex451@gmail.com>

* add model attribute to connector

Signed-off-by: ericharper <complex451@gmail.com>

* add model attribute to connector

Signed-off-by: ericharper <complex451@gmail.com>

* fix tabs

Signed-off-by: ericharper <complex451@gmail.com>

* fix tabs

Signed-off-by: ericharper <complex451@gmail.com>

* remove ModelPT import

Signed-off-by: ericharper <complex451@gmail.com>

* add default restore

Signed-off-by: ericharper <complex451@gmail.com>

* add default restore

Signed-off-by: ericharper <complex451@gmail.com>

* remove eff globals

Signed-off-by: ericharper <complex451@gmail.com>

* style

Signed-off-by: ericharper <complex451@gmail.com>

* fix tabs

Signed-off-by: ericharper <complex451@gmail.com>

* style

Signed-off-by: ericharper <complex451@gmail.com>

* update globals, remove save restore property

Signed-off-by: ericharper <complex451@gmail.com>

* fix typo

Signed-off-by: ericharper <complex451@gmail.com>

* add setter

Signed-off-by: ericharper <complex451@gmail.com>

* add setter

Signed-off-by: ericharper <complex451@gmail.com>

* fix app_state restore flag

Signed-off-by: ericharper <complex451@gmail.com>

* typo

Signed-off-by: ericharper <complex451@gmail.com>

* update paths

Signed-off-by: ericharper <complex451@gmail.com>

* add connector arg to from_pretrained

Signed-off-by: ericharper <complex451@gmail.com>

* update save restore connector after instantiating

Signed-off-by: ericharper <complex451@gmail.com>

* use connector

Signed-off-by: ericharper <complex451@gmail.com>

* get class from config in .nemo

Signed-off-by: ericharper <complex451@gmail.com>

* add TODO

Signed-off-by: ericharper <complex451@gmail.com>

* move extract_state_dict to connector

Signed-off-by: ericharper <complex451@gmail.com>

* add methods for toch save and torch load

Signed-off-by: ericharper <complex451@gmail.com>

* typo

Signed-off-by: ericharper <complex451@gmail.com>

* typo

Signed-off-by: ericharper <complex451@gmail.com>

* typo

Signed-off-by: ericharper <complex451@gmail.com>

* update mock model conf

Signed-off-by: ericharper <complex451@gmail.com>

* revert

Signed-off-by: ericharper <complex451@gmail.com>

* move mock model to common collection

Signed-off-by: ericharper <complex451@gmail.com>

* update NLPModel

Signed-off-by: ericharper <complex451@gmail.com>

* update test to use connector

Signed-off-by: ericharper <complex451@gmail.com>

* move artifacts to save restore connector

Signed-off-by: ericharper <complex451@gmail.com>

* add save_restore_connector arg to register_artifact

Signed-off-by: ericharper <complex451@gmail.com>

* style

Signed-off-by: ericharper <complex451@gmail.com>

* default save_restore_connector arg to None

Signed-off-by: ericharper <complex451@gmail.com>

* default save_restore_connector arg to None

Signed-off-by: ericharper <complex451@gmail.com>

* clean commented line

Signed-off-by: ericharper <complex451@gmail.com>

* default save_restore_connector arg to None

Signed-off-by: ericharper <complex451@gmail.com>

* move MockModel

Signed-off-by: ericharper <complex451@gmail.com>

* fix docstrings, remove underscores, default from connector

Signed-off-by: ericharper <complex451@gmail.com>

* update docstring

Signed-off-by: ericharper <complex451@gmail.com>

* update docstring

Signed-off-by: ericharper <complex451@gmail.com>

* change name to is_model_being_restored

Signed-off-by: ericharper <complex451@gmail.com>

* move constants from AppState to SaveRestoreConnector

Signed-off-by: ericharper <complex451@gmail.com>

* encapsulate logic for model parallel checkpoint

Signed-off-by: ericharper <complex451@gmail.com>

* style

Signed-off-by: ericharper <complex451@gmail.com>

* update mock config

Signed-off-by: ericharper <complex451@gmail.com>

* remove unused import

Signed-off-by: ericharper <complex451@gmail.com>

* add init_subclass, remove connector arg from register_artifact, move MockModel to tests

Signed-off-by: ericharper <complex451@gmail.com>

* remove old import

Signed-off-by: ericharper <complex451@gmail.com>

* Add tests

Signed-off-by: smajumdar <titu1994@gmail.com>

* Finalize tests

Signed-off-by: smajumdar <titu1994@gmail.com>

* Finalize tests

Signed-off-by: smajumdar <titu1994@gmail.com>

* fixing lgtm

Signed-off-by: ericharper <complex451@gmail.com>

* fix lgtm

Signed-off-by: ericharper <complex451@gmail.com>

* update NLPModel.restore_from

Signed-off-by: ericharper <complex451@gmail.com>

* Fix classpath resolution

Signed-off-by: smajumdar <titu1994@gmail.com>

Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
 2021-08-05 17:45:57 -07:00
Boris Fomitchev 17b68d73c9
CitriNet export fix (#2620) 
* Fixing CitriNet ONNX export

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

* Reverting context_window back to int

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

* Cleaning up extra script

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
 2021-08-05 16:50:48 -07:00
Tuan Manh Lai b0c9afb4e8
Allow training a multilingual duplex TN model (#2583) 
* Allow training a multilingual duplex TN model
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Add copyright header to combine_processed_datasets.py
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Minor Fix
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com>
 2021-08-05 15:52:39 -07:00
Tuan Manh Lai 3412ee0b44
Remove unused arg in TextNormalizationTestDataset (#2613) 
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com>
 2021-08-05 11:27:59 -07:00
QIliang 7426a9d60b
resolved MisconfigurationException (#2616) 
end of epoch will throw execption:

pytorch_lightning.utilities.exceptions.MisconfigurationException: ModelCheckpoint(monitor='val_loss') not found in the returned metrics: ['loss', 'l_mle', 'l_length', 'logdet']. HINT: Did you call self.log('val_loss', value) in the LightningModule?


Signed-off-by: Qiliang <xiaoqlster@gmail.com>
 2021-08-05 10:07:29 -04:00
Somshubra Majumdar f092c7f656
Make ITN tests optional (run only on change) (#2611) 
* Make ITN tests optional run only on change

Signed-off-by: smajumdar <titu1994@gmail.com>

* Revert Jenkinsfile for RNNT test

Signed-off-by: smajumdar <titu1994@gmail.com>
 2021-08-04 10:48:33 -07:00
Somshubra Majumdar 7051487c7e
Update contextnet configs (#2601) 
Signed-off-by: smajumdar <titu1994@gmail.com>
 2021-08-03 15:01:04 -07:00
Evelina 97e67f7790
import fix (#2597) 
* import fix

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* remove unused import

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* example import fix

Signed-off-by: ekmb <ebakhturina@nvidia.com>
 2021-08-03 12:03:14 -07:00
Somshubra Majumdar d04c7e9b4e
Integrate NVIDIA DALI 1.4 to NeMo ASR (#2567) 
* Initial prototype of ASR DALI integration with DALI 1.4

Signed-off-by: smajumdar <titu1994@gmail.com>

* Update dali support to 1.4

Signed-off-by: smajumdar <titu1994@gmail.com>

* Fix docs

Signed-off-by: smajumdar <titu1994@gmail.com>

* Address comments

Signed-off-by: smajumdar <titu1994@gmail.com>

* Apply suggestions from code review

Co-authored-by: Janusz Lisiecki <39967756+JanuszL@users.noreply.github.com>

* Address comments

Signed-off-by: smajumdar <titu1994@gmail.com>

* Correct module utils

Signed-off-by: smajumdar <titu1994@gmail.com>

Co-authored-by: Janusz Lisiecki <39967756+JanuszL@users.noreply.github.com>
 2021-08-03 11:01:51 -07:00
Yang Zhang dfb5bf5b74
asr gemran (#2590) 
Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
 2021-08-02 10:49:54 -07:00
Tuan Manh Lai 2133e7e834
Fixes for neural TN (#2581) 
* Preprocessed Google TN data not need basic tokenization
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Minor Fix
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Error fix
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Added test after train
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Minor Fix
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Allow data caching for Tagger Dataset
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Allow data caching for Decoder dataset
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Minor fixes
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com>
 2021-07-30 14:52:58 -07:00
Eric Harper 4a7c3e3df4
Merge 1.2 bugfixes into main (#2588) 
* update jenkinsfile

Signed-off-by: ericharper <complex451@gmail.com>

* update BRANCH

Signed-off-by: ericharper <complex451@gmail.com>

* Fix onnx for ASR notebook (#2542)

* Update onnx version

Signed-off-by: smajumdar <titu1994@gmail.com>

* Fix onnx

Signed-off-by: smajumdar <titu1994@gmail.com>

* Fix onnx

Signed-off-by: smajumdar <titu1994@gmail.com>

* Fix typos and MeCab import (#2541)

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix branch for ASR notebooks (#2549)

Signed-off-by: smajumdar <titu1994@gmail.com>

* rmtok (#2559)

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* Add xxhash dependency (#2564)

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix (#2566)

* fix

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* doc add

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* style fix

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix moses path issue (#2573)

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* More moses data path fixes (#2575)

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Path fixes (#2580)

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Upper bound transformers for 1.2 (#2584)

* upper bound transformers and name change jarvis to riva

Signed-off-by: ericharper <complex451@gmail.com>

* upper bound transformers and name change jarvis to riva

Signed-off-by: ericharper <complex451@gmail.com>

* update jenkinsfile

Signed-off-by: ericharper <complex451@gmail.com>

* update notebooks branch

Signed-off-by: ericharper <complex451@gmail.com>

* update notebooks branch

Signed-off-by: ericharper <complex451@gmail.com>

* update notebooks branch

Signed-off-by: ericharper <complex451@gmail.com>

Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
 2021-07-30 11:53:07 -06:00
PeganovAnton b5b29a69cc
Fix manifest name (#2546) 
Signed-off-by: PeganovAnton <peganoff2@mail.ru>

Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
 2021-07-27 11:00:58 -07:00
Yang Zhang eaa2b1707b
add seconds to time for english tn (#2550) 
Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
 2021-07-27 07:34:10 -07:00
Aleksei Kalinov 92c79faf5f
Remove deprecated import from source file. (#2481) 
Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
 2021-07-26 22:51:26 -07:00
Yang Zhang 222411fb71
fix bug (#2527) 
Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
 2021-07-26 21:22:26 -07:00
Jason cd6a691607
Style Patch (#2556) 
* Update audio_preprocessing.py

* Update audio_preprocessing.py

Signed-off-by: Jason <jasoli@nvidia.com>

* style

Signed-off-by: Jason <jasoli@nvidia.com>
 2021-07-26 10:51:43 -04:00
Jason f2910cb01d
Update audio_preprocessing.py (#2538) 
* Update audio_preprocessing.py

* Update audio_preprocessing.py

Signed-off-by: Jason <jasoli@nvidia.com>
 2021-07-26 09:03:55 -04:00
Ghasem bb7a9304fa
Adding Non English Tutor (#2532) 
* Adding Non English Tutor

Signed-off-by: Ghasem Pasandi <gpasandi@nvidia.com>

* applying Evelina's comments

Signed-off-by: Ghasem Pasandi <gpasandi@nvidia.com>

Co-authored-by: Ghasem Pasandi <gpasandi@nvidia.com>
 2021-07-23 09:02:12 -07:00
Eric Harper 0349415662
Merge bugfixes and doc updates from 1.2.0 to main (#2533) 
* update jenkinsfile

Signed-off-by: ericharper <complex451@gmail.com>

* update BRANCH

Signed-off-by: ericharper <complex451@gmail.com>

* update package_info.py

Signed-off-by: ericharper <complex451@gmail.com>

* Update Dockerfile numba install for 21.06 (#2515)

* update Dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* update Dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* upper bound ptl 1.4 (#2517)

Signed-off-by: ericharper <complex451@gmail.com>

* Typo correction in asr streaming tutorial (#2520)

* Corrected typos

Signed-off-by: jbalam <jbalam@nvidia.com>

* datalayer->data layer

Signed-off-by: jbalam <jbalam@nvidia.com>

* Jarvis to Riva changes for doc 1.2.0 (#2521)

* Change Jarvis to Riva in export.rst (#2529)

Signed-off-by: Herb Kelly <hkelly@nvidia.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update version

Signed-off-by: ericharper <complex451@gmail.com>

Co-authored-by: Jagadeesh Balam <4916480+jbalam-nv@users.noreply.github.com>
Co-authored-by: hkelly33 <58792115+hkelly33@users.noreply.github.com>
 2021-07-22 09:31:38 -06:00
Aleksandr Laptev a367249872
Convert kaldi data folder to manifest.json (#2447) 
* kaldi data folder to manifest converting

Signed-off-by: GNroy <laptevsasha12@gmail.com>

* pipe and offset support

Signed-off-by: GNroy <laptevsasha12@gmail.com>

* logging added

Signed-off-by: GNroy <laptevsasha12@gmail.com>

* apply style fix

Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Co-authored-by: Aleksandr Laptev <alaptev@nvidia.com>
 2021-07-21 17:04:36 -07:00
Nithin Rao 9fad1dc929
Fix time stamps (#2522) 
* time stamps done

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* multi batch_size support

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* add doc strings

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* spelling fix

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* bs default

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* test jenkins remove file

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* revert file not found jenkins fix

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* subsegments rename

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* out_dir check

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
 2021-07-21 14:42:29 -07:00
Vahid Noroozi 57406c3975
Support for char-level models. (#2530) 2021-07-21 12:12:58 -07:00
Tuan Manh Lai b472670afa
Extending the neural TN/ITN models for other languages (#2497) 
* extending the neural TN/ITN model to handle RU

Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Support German
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Catch AttributeError instead of BaseException
Signed-off-by: Tuan Lai <tuanl@nvidia.com>

* Style fix
Signed-off-by: Tuan Lai <tuanl@nvidia.com>
 2021-07-21 09:22:53 -07:00
Somshubra Majumdar dc6960509f
Enable RNNT ONNX Export (#2510) 
* Begin export of RNNT models

Signed-off-by: smajumdar <titu1994@gmail.com>

* Prepare export

Signed-off-by: smajumdar <titu1994@gmail.com>

* Refactor Exportable

Signed-off-by: smajumdar <titu1994@gmail.com>

* Enable RNNT export to onnx

Signed-off-by: smajumdar <titu1994@gmail.com>

* RNNT export

Signed-off-by: smajumdar <titu1994@gmail.com>

* Attempt to get stateful inference

Signed-off-by: smajumdar <titu1994@gmail.com>

* Revert changes attempt state fix

Signed-off-by: smajumdar <titu1994@gmail.com>

* Hack together forward pass <FIX LATER>

Signed-off-by: smajumdar <titu1994@gmail.com>

* Return length

Signed-off-by: smajumdar <titu1994@gmail.com>

* Correct signature

Signed-off-by: smajumdar <titu1994@gmail.com>

* Prepare stateful

Signed-off-by: smajumdar <titu1994@gmail.com>

* Continie checks for states

Signed-off-by: smajumdar <titu1994@gmail.com>

* Correct naming

Signed-off-by: smajumdar <titu1994@gmail.com>

* Correct names

Signed-off-by: smajumdar <titu1994@gmail.com>

* Pass trace checks

Signed-off-by: smajumdar <titu1994@gmail.com>

* Update test to be dynamic

Signed-off-by: smajumdar <titu1994@gmail.com>

* Update naming semantic

Signed-off-by: smajumdar <titu1994@gmail.com>

* Initial prototype of RNNT greedy decoding

Signed-off-by: smajumdar <titu1994@gmail.com>

* Fix tracer warnings

Signed-off-by: smajumdar <titu1994@gmail.com>

* Corrected names of outputs for RNNT encoder

Signed-off-by: smajumdar <titu1994@gmail.com>

* Fix export rnnt script

Signed-off-by: smajumdar <titu1994@gmail.com>

* Prototype RNNT forward inference

Signed-off-by: smajumdar <titu1994@gmail.com>

* Update asr export

Signed-off-by: smajumdar <titu1994@gmail.com>

* Runnable inference

Signed-off-by: smajumdar <titu1994@gmail.com>

* Correct log softmax for joint

Signed-off-by: smajumdar <titu1994@gmail.com>

* Finish RNNT export to onnx

Signed-off-by: smajumdar <titu1994@gmail.com>

* Fix onnx export

Signed-off-by: smajumdar <titu1994@gmail.com>

* Create parallel version of ConvASREncoder (#2456)

* Add parallel block that supports Jasper-like blocks.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Add Encoder with parallel blocks.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Mark test as unit test.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Support pointwise residual connections to connect layers with different fitlers.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Format source code

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Switch to 1D tensor for dropout weights.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Format source code.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>

* Fix style

Signed-off-by: smajumdar <titu1994@gmail.com>

* Update comments

Signed-off-by: smajumdar <titu1994@gmail.com>

* Freeze the nemo model

Signed-off-by: smajumdar <titu1994@gmail.com>

* Update transcribe script to support RNNT and rnnt model transcriptions

Signed-off-by: smajumdar <titu1994@gmail.com>

* Add threshold

Signed-off-by: smajumdar <titu1994@gmail.com>

* Add warning

Signed-off-by: smajumdar <titu1994@gmail.com>

* Minor fixes

Signed-off-by: smajumdar <titu1994@gmail.com>

* Style fix

Signed-off-by: smajumdar <titu1994@gmail.com>

* Correctly replace forward step

Signed-off-by: smajumdar <titu1994@gmail.com>

* Reset RNNT flag after export is done

Signed-off-by: smajumdar <titu1994@gmail.com>

* Generalize _rnnt_export flag

Signed-off-by: smajumdar <titu1994@gmail.com>

* Generalize _rnnt_export flag

Signed-off-by: smajumdar <titu1994@gmail.com>

Co-authored-by: Aleksei Kalinov <alekseia@nvidia.com>
 2021-07-20 20:01:18 -07:00
Evelina c8f9427295
Eng TN update (#2516) 
* added url support

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* address added

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* sh test and export update

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* sh test and export update

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* fix fraction for sh

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* telephone with words added

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* remove unused import

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* clean up

Signed-off-by: ekmb <ebakhturina@nvidia.com>

* update

Signed-off-by: ekmb <ebakhturina@nvidia.com>
 2021-07-20 14:57:58 -07:00
Yang Zhang a8b6a1a4dd
Itn german (#2486) 
* initial german itn

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* cardinal now can accept compound, hundred and thousand without any prefix, ordinal verbalization deleted suffix and replaced with dot

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added all date options

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added fraction

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added fraction to measure

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* default cent to euro

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added fraction tsv (forgot in the past) and added hour to night to time class

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* adjusted docstring to german

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix lgtm

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* delete unnecessary copyright

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix header and delete wrong spelling

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* delete wrong spelling

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added missing data values for time

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* delete SH normalization test, cause it doesnt exist

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added all classes to SH test

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* deleted redundant files, updated header

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* adding back whitelist

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
 2021-07-20 12:41:36 -07:00
Micha Livne d08c7a7b9c
Nmt byte level tokenizer model fix (#2421) 
* 1. Updated permissions to include x.

* 1. Fixed byte-level tokenizer when no model for tokenizer is given.

Signed-off-by: Micha Livne <mlivne@nvidia.com>

Co-authored-by: Micha Livne <mlivne@nvidia.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
 2021-07-19 17:12:36 -07:00
Jason 846b150082
Update TTS Docs to recommend fastpitch and hifigan (#2498) 
* update docs

Signed-off-by: Jason <jasoli@nvidia.com>

* update

Signed-off-by: Jason <jasoli@nvidia.com>
 2021-07-19 16:30:35 -07:00
David ffb80e1bbd
minor updates (#2465) 
Signed-off-by: David Mosallanezhad <amosalla@asu.edu>

Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>
 2021-07-19 16:29:52 -07:00
roman-vygon 99557a6f6e
apply fix (#2461) 
Signed-off-by: roman-vygon <roman.vygon@gmail.com>

Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>
 2021-07-19 16:13:45 -07:00
vadam5 e3f6867dd2
Entity linking documentation (#2357) 
* Update tutorials.rst

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update tutorials.rst

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update models.rst

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Add files via upload

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Create entity_linking.rst

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update README.rst

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update entity_linking.rst

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update nlp_all.bib

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update entity_linking.rst

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update entity_linking.rst

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* fixed base typos and doc link

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
 2021-07-19 16:10:19 -07:00
khcs 9db4053e07
Update Relation_Extraction-BioMegatron.ipynb (#2389) 
fix 'DDP' to 'ddp'

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
 2021-07-19 16:07:33 -07:00
Eric Harper c527e954b6
Update container version to 21.06 (#2431) 
* update container version

Signed-off-by: ericharper <complex451@gmail.com>

* remove conda update from reinstall.sh

Signed-off-by: ericharper <complex451@gmail.com>

* pin numba in reinstall

Signed-off-by: ericharper <complex451@gmail.com>
 2021-07-16 11:51:53 -07:00
felixmcgregor e5b8570eec
Remove multiplication of sample rate for pydub indexing (#2482) 
Signed-off-by: fmcgregor <felix@saigen.co.za>

Co-authored-by: fmcgregor <felix@saigen.co.za>
Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
 2021-07-16 09:11:08 -07:00
Aleksei Kalinov f893721684
Create parallel version of ConvASREncoder (#2456) 
* Add parallel block that supports Jasper-like blocks.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Add Encoder with parallel blocks.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Mark test as unit test.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Support pointwise residual connections to connect layers with different fitlers.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Format source code

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Switch to 1D tensor for dropout weights.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

* Format source code.

Signed-off-by: Aleksei Kalinov <alekseia@nvidia.com>

Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
 2021-07-16 16:22:47 +03:00
Yang Zhang 159952d71f
add sgdqa to readme (#2492) 
Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

Co-authored-by: Eric Harper <complex451@gmail.com>
 2021-07-15 23:14:32 -06:00
Eric Harper 765920cd68
Update README.rst 2021-07-15 18:17:19 -06:00
Sandeep Subramanian 3992946be1
Minor fixes to NMT Data Preprocessing Notebook (#2491) 
* Minor fixes to NMT data notebok

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* One more fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* One more fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
 2021-07-15 15:01:25 -07:00
Jason d445f44dfd
Add copyright headers check (#2490) 
* add cpr hdrs

Signed-off-by: Jason <jasoli@nvidia.com>

* update

Signed-off-by: Jason <jasoli@nvidia.com>

* review

Signed-off-by: Jason <jasoli@nvidia.com>
 2021-07-15 15:06:45 -04:00