text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
// Copyright (C) 2020 Mathieu Duponchelle <mathieu@centricular.com>
|
|
|
|
//
|
2022-01-15 18:40:12 +00:00
|
|
|
// This Source Code Form is subject to the terms of the Mozilla Public License, v2.0.
|
|
|
|
// If a copy of the MPL was not distributed with this file, You can obtain one at
|
|
|
|
// <https://mozilla.org/MPL/2.0/>.
|
text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
//
|
2022-01-15 18:40:12 +00:00
|
|
|
// SPDX-License-Identifier: MPL-2.0
|
text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
|
2021-06-03 18:20:54 +00:00
|
|
|
use gst::prelude::*;
|
text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
|
|
|
|
fn init() {
|
|
|
|
use std::sync::Once;
|
|
|
|
static INIT: Once = Once::new();
|
|
|
|
|
|
|
|
INIT.call_once(|| {
|
|
|
|
gst::init().unwrap();
|
|
|
|
gstrsregex::plugin_register_static().expect("regex test");
|
|
|
|
});
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn test_replace_all() {
|
|
|
|
init();
|
|
|
|
|
|
|
|
let input = b"crap that mothertrapper";
|
|
|
|
|
|
|
|
let expected_output = "trap that mothertrapper";
|
|
|
|
|
|
|
|
let mut h = gst_check::Harness::new("regex");
|
|
|
|
|
|
|
|
{
|
2021-04-12 12:49:54 +00:00
|
|
|
let regex = h.element().expect("Could not create regex");
|
text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
|
2021-11-06 07:34:10 +00:00
|
|
|
let command = gst::Structure::builder("replace-all")
|
|
|
|
.field("pattern", "crap")
|
|
|
|
.field("replacement", "trap")
|
|
|
|
.build();
|
text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
|
2021-11-06 07:34:10 +00:00
|
|
|
let commands = gst::Array::from(vec![command.to_send_value()]);
|
text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
|
2021-11-08 09:55:40 +00:00
|
|
|
regex.set_property("commands", &commands);
|
text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
h.set_src_caps_str("text/x-raw, format=utf8");
|
|
|
|
|
|
|
|
let buf = {
|
|
|
|
let mut buf = gst::Buffer::from_mut_slice(Vec::from(&input[..]));
|
|
|
|
let buf_ref = buf.get_mut().unwrap();
|
2022-10-17 17:48:43 +00:00
|
|
|
buf_ref.set_pts(gst::ClockTime::ZERO);
|
|
|
|
buf_ref.set_duration(2.seconds());
|
text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
buf
|
|
|
|
};
|
|
|
|
|
|
|
|
assert_eq!(h.push(buf), Ok(gst::FlowSuccess::Ok));
|
|
|
|
|
|
|
|
let buf = h.pull().expect("Couldn't pull buffer");
|
|
|
|
|
2021-05-26 17:03:09 +00:00
|
|
|
assert_eq!(buf.pts(), Some(gst::ClockTime::ZERO));
|
2022-10-17 17:48:43 +00:00
|
|
|
assert_eq!(buf.duration(), Some(2.seconds()));
|
text: new element for text processing: regex
The element expects an array of "commands", as GstStructures,
in the form:
operation, pattern=<pattern>, ...
The only operation implemented for now is replace-all, eg:
replace-all, pattern=foo, replacement=bar
Other operations can be implemented if useful in the future,
eg. "match" could post a message to the bus when the pattern
is encountered.
The main use case for this is automatic speech recognition,
as implemented by eg awstranscribe as users may want to replace
swear words with tamer language.
Commands are applied in order.
The interface is usable through the CLI with the usual escaping
strategies, though trying to pass in actual regular expressions
through it is a bit tricky, as this introduces yet another
level of escaping.
2021-02-18 23:45:34 +00:00
|
|
|
|
|
|
|
let map = buf.map_readable().expect("Couldn't map buffer readable");
|
|
|
|
|
|
|
|
assert_eq!(
|
|
|
|
std::str::from_utf8(map.as_ref()),
|
|
|
|
std::str::from_utf8(expected_output.as_ref())
|
|
|
|
);
|
|
|
|
}
|