oss/starmelon
oss/starmelon
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore: upgrade deno_* crates to Mar 2024

Browse source 
The way extensions are injected into the Isolate changed dramatically
between Sept 2021 and Jan 2024. Now ops are available in the virtual
module ext:core/ops.
This commit is contained in:
nobody 2024-04-17 17:18:04 -07:00
commit ac97ac37db
Signed by: GrocerPublishAgent
GPG key ID: D460CD54A9E3AB86
23 changed files with 618 additions and 303 deletions
Download patch file Download diff file Expand all files Collapse all files

21
Cargo.toml
View file

@ -5,49 +5,46 @@ edition = "2018"
# I decided to use os_pipe becaues I want to pipe stdout of a subprocess into # I decided to use os_pipe becaues I want to pipe stdout of a subprocess into
# stderr in real time. I want the output of the process to stderr and stdout # stderr in real time. I want the output of the process to stderr and stdout
# show up in the original order. I looked an os_pipe uses some unsafe code to # show up in the original order. I looked and os_pipe uses some unsafe code to
# duplicate file descriptors. I am unfamiliar with the correct way of sharing # duplicate file descriptors. I am unfamiliar with the correct way of sharing
# the fds. Therefore I am going to trust that their unsafe code is necessary. # the fds. Therefore I am going to trust that their unsafe code is necessary.
# Then it makes sense to use a battle tested unsafe code rather than implement # Then it makes sense to use a battle tested unsafe code rather than implement
# it myself. # it myself.
[dependencies] [dependencies]
ahash = "0.7" ahash = "0.8"
clap = { version = "4.5", features = ["derive"] }
elmi = { path = "../../../infra/rust-elmi", features = [ "genco" ] } elmi = { path = "../../../infra/rust-elmi", features = [ "genco" ] }
naive-wadler-prettier= { path = "../../../infra/redwood-lang/compiler/naive-wadler-prettier" } naive-wadler-prettier= { path = "../../../infra/redwood-lang/compiler/naive-wadler-prettier" }
os_pipe = "1.0" os_pipe = "1.0"
serde = { version = "1.0", features = [ "derive" ] } serde = { version = "1.0", features = [ "derive" ] }
serde_json = { version ="1.0", features = [] } serde_json = { version ="1.0", features = [] }
structopt = { version = "0.3" }
elm-project-utils = { path = "../../../infra/rust-elm-project-utils" } elm-project-utils = { path = "../../../infra/rust-elm-project-utils" }
tracing = { version = "0.1", features = [] } tracing = { version = "0.1", features = [] }
rustc-hash = "1.1" rustc-hash = "1.1"
home = "0.5" home = "0.5"
# Required to transpile view functions to Rust # Required to transpile view functions to Rust
genco = "0.17" genco = "0.17"
# Required to generate fixture Elm files # Required to generate fixture Elm files
genco-extra = { path = "../../../infra/genco-extra" } genco-extra = { path = "../../../infra/genco-extra" }
# All of these are required for deno's javascript runtime. We need to keep the # All of these are required for deno's javascript runtime. We need to keep the
# same versions as other projects in our cargo workspace. Multiple different # same versions as other projects in our cargo workspace. Multiple different
# versions of rusty_v8 seem to break its build script. # versions of rusty_v8 seem to break its build script.
deno_runtime = "0.127" deno_runtime = "0.154"
tokio = { version = "1.32", features = ["full"] } tokio = { version = "1.32", features = ["full"] }
deno_core = "0.214" deno_core = "0.272"
deno_web = "0.150" deno_web = "0.177"
futures = "0.3" futures = "0.3"
# Required to add sql query support to interpreter. Because deno expects sync # Required to add sql query support to interpreter. Because deno expects sync
# ops to be synchronous, we have to use a second async executor to run the sqlx # ops to be synchronous, we have to use a second async executor to run the sqlx
# functions. I read the code for oneshot # functions. I read the code for oneshot
# (https://github.com/faern/oneshot/commit/9aa237f185e1b65d61bf92c20350cf7bee0aa88b) # (https://github.com/faern/oneshot/commit/9aa237f185e1b65d61bf92c20350cf7bee0aa88b)
# and it looks reasonable. # and it looks reasonable.
oneshot = { version = "0.1.6", features = ["std"]}
sqlx = { version = "0.7", features = [ "sqlite", "macros", "runtime-tokio", "chrono", "json", "uuid" ] } sqlx = { version = "0.7", features = [ "sqlite", "macros", "runtime-tokio", "chrono", "json", "uuid" ] }
oneshot = "0.1.3"
# required for livetable derive macro # required for livetable derive macro
livetable-core = { path = "../../../infra/livetable/core" } livetable-core = { path = "../../../infra/livetable/core" }
# required for clean subcommand
walkdir = "2.4"
[dev-dependencies] [dev-dependencies]

49
README.md Normal file
View file

@ -0,0 +1,49 @@
When I first created starmelon in September 2021 there was a hacky workaround
for setTimeout I had to inject into the generated javascript. Now in Nov 2023
after upgrading deno to 0.127 I find that `globalThis.__boostrap.timers` is no
longer defined. Now the elm code seems to work without requring this hack.
However I don't have any correctness tests for starmelon. To reduce the risk of
forgetting how it was working I have included the hacks from
src/exec/scripting.rs:168
```rust
// I think that when I set this script to be the main module, I am skipping the
// deno/runtime/js/99_main.js script that sets up a bunch of global variables. If I
// manually add the timer related code below then setTimeout works again.
// NB. there are 706 lines of setup code that add a bunch of apis to the global window
// scope. Figure out if I need to include all of them. For example, starmelon does not need
// to perform http calls right now, but I eventually want to.
final_script.push_str("const { setTimeout } = globalThis.__bootstrap.timers;\n");
final_script.push_str(
"Deno.core.setMacrotaskCallback(globalThis.__bootstrap.timers.handleTimerMacrotask);\n",
);
final_script.push_str("globalThis.setTimeout = setTimeout;\n");
```
Somewhere between `deno_runtime` version 0.127 and 0.147 they decided to remove
`deno_core::op` macro and replace it with `deno_core::op2`. As far as I can
tell, the `op2` macro offers greater control of how values are passed between
Rust and JavaScript.
In Jan 2024 they moved operations to a virtual module
`import { op_example } from "ext:core/ops`. You can only access this virtual
module in the bootstrap ESM scripts of an `deno_core::Extension`. Consequently
what I have done is write a bootstrap script that imports the ops and reassigns
them as properties of `globalThis.Extension` object. All of my extensions are
merged onto the same object. It appears the Deno.core is deleted by the time
the `deno_runtime::worker::MainWorker` runs the main module.
`Deno[Deno.internal].core.ops` still exists but does not contain the ops our
Extensions defined.
An aside is that if you construct a JsRuntime directly and add Extensions then
those ops will show up on `Deno.core.ops`. But they will not be enumerable
properties, so you will have to use `Object.getOwnPropertyNames(Deno.core.ops)`
to visually confirm the ops are there. My current understanding in April 2024
is that MainWorker includes a JsRuntime, but then also applies all of the
`deno_runtime` extensions that make the JS enviroment feature comparable with
Node. For example setTimeout does not exist in a new JsRuntime but does exist
in a new MainWorker.
You can find some of the extensions that Deno provides in
[https://github.com/denoland/deno/tree/main/runtime/js]

2
always-rebuild.do
View file

@ -1,2 +0,0 @@
redo-always
redo-ifchange release

1
examples/sqlite-integration/README.md Normal file
View file

@ -0,0 +1 @@
./starmelon exec --sqlite example.sqlite src/ShowDatabaseSchema.elm main --output=schema.html

2
examples/sqlite-integration/src/ShowDatabaseSchema.elm
View file

@ -11,7 +11,7 @@ main =
let let
query = query =
fetch fetch
"select json_object('type', type, 'name', name, 'sql', sql) from sqlite_master WHERE type = 'table'" "select type, name, sql from sqlite_master WHERE type = 'table'"
[] []
(Json.Decode.map3 (Json.Decode.map3
(\kind name sql-> (\kind name sql->

2
release.do
View file

@ -1,4 +1,4 @@
redo-ifchange Cargo.toml redo-ifchange Cargo.toml
find src/ -type f | xargs redo-ifchange find src/ -type f | xargs redo-ifchange
cargo build --release cargo build --color=always --release

96
src/debug.rs Normal file
View file

@ -0,0 +1,96 @@
use deno_core::{Extension, OpState, op2, extension};
use elm_project_utils::{setup_generator_project, ElmResult};
use os_pipe::dup_stderr;
use sqlx::sqlite::SqlitePool;
use std::{
borrow::Cow,
cell::RefCell,
convert::TryFrom,
fs,
io::{self, Read, Write},
path::PathBuf,
process::{Command, Stdio},
sync::Arc,
};
use tracing::info_span;
use crate::exec::{fixtures, runtime};
use crate::reporting::{CompilerError, InterpreterError, Problem, TypeError};
#[deno_core::op2(fast)]
fn op_starmelon_example(
state: &mut OpState,
#[string] msg: String,
) -> Result<(), deno_core::error::AnyError> {
eprintln!("got a String message from v8 runtime {:?}", &msg);
Ok(())
}
#[op2(fast)]
fn op_hello(#[string] text: &str) {
println!("Hello {} from an op!", text);
}
extension!(
debug_ext,
ops = [op_hello],
esm_entry_point = "ext:debug_ext/debug_bootstrap.js",
esm = [dir "src", "debug_bootstrap.js"],
docs = " small example demonstrating op2 usage.", "Contains one op."
);
pub(crate) fn run(javascript_path: Option<PathBuf>) -> Result<(), Problem> {
let final_path: PathBuf = if let Some(ref path) = javascript_path {
std::fs::canonicalize(path).unwrap()
//std::fs::read_to_string(&path)
// .map_err(|io_err| CompilerError::ReadInputFailed(io_err, "stdout".into()))?
} else {
return Ok(());
//"console.log('hello world')".to_owned()
};
// Create a tokio runtime before registering ops so we can block on futures inside sync ops
let span = info_span!("create tokio runtime");
let timing_guard = span.enter();
let sys = tokio::runtime::Builder::new_current_thread()
// The default number of additional threads for running blocking FnOnce is 512.
.max_blocking_threads(1)
.enable_all()
.build()
.unwrap();
drop(timing_guard);
// step 10 create a v8 isolate. We need to register a different callback depending on
// the output type (string, or bytes)
//let mailbox: Arc<RefCell<Option<Result<Vec<u8>, String>>>> = Arc::new(RefCell::new(None));
//let mailbox_clone = Arc::clone(&mailbox);
let mut extensions = vec![
Extension {
ops: Cow::Owned(vec![op_starmelon_example::decl()]),
op_state_fn: Some(Box::new(move |state| {
//state.put(Arc::clone(&mailbox_clone));
})),
..Default::default()
},
debug_ext::init_ops_and_esm(),
];
let span = info_span!("create v8 isolate");
let timing_guard = span.enter();
let (worker, main_module) = runtime::setup_worker(extensions, &final_path.to_string_lossy())
.map_err(|err| InterpreterError::EventLoop(err))?;
drop(timing_guard);
let entrypoint =
move |mut scope: deno_core::v8::HandleScope| -> Result<(), InterpreterError> { Ok(()) };
let span = info_span!("eval javascript");
let timing_guard = span.enter();
sys.block_on(async move { runtime::xyz(worker, main_module, entrypoint).await })?;
drop(timing_guard);
Ok(())
}

7
src/debug_bootstrap.js Normal file
View file

@ -0,0 +1,7 @@
// Copyright 2018-2024 the Deno authors. All rights reserved. MIT license.
import { op_hello } from "ext:core/ops";
function hello() {
op_hello("world");
}
globalThis.Extension = { hello };

4
src/derive/mod.rs
View file

@ -180,8 +180,8 @@ fn resolve_table_argument(tipe: &elmi::Type) -> Result<i32, TypeError> {
//ColumnType::MaybeFloat => Maybe Float, //ColumnType::MaybeFloat => Maybe Float,
//ColumnType::Double => Float, //ColumnType::Double => Float,
//ColumnType::MaybeDouble => Maybe Float, //ColumnType::MaybeDouble => Maybe Float,
//ColumnType::Timestamp => #(posix_type.clone()), //ColumnType::Timestamp => $(posix_type.clone()),
//ColumnType::MaybeTimestamp => Maybe #(posix_type.clone()), //ColumnType::MaybeTimestamp => Maybe $(posix_type.clone()),
//ColumnType::ExactlyOneAttachment => LiveTable.Attachment, //ColumnType::ExactlyOneAttachment => LiveTable.Attachment,
//ColumnType::MaybeAttachment => Maybe LiveTable.Attachment, //ColumnType::MaybeAttachment => Maybe LiveTable.Attachment,
//ColumnType::ListAttachment => List LiveTable.Attachment, //ColumnType::ListAttachment => List LiveTable.Attachment,

1
src/elm.rs
View file

@ -30,6 +30,7 @@ pub fn make(file: &Path, debug: bool, verbosity: u64) -> Result<(), Problem> {
} }
} }
Err(_io_err) => { Err(_io_err) => {
eprintln!("{:?}", _io_err);
// TODO handle this one // TODO handle this one
return Err(Problem::Wildcard("elm failed".into())); return Err(Problem::Wildcard("elm failed".into()));
} }

93
src/exec/astrid_pages.rs
View file

@ -1,22 +1,24 @@
use crate::exec::fixtures::astrid_pages::ScriptError; use deno_core::{v8, Extension, OpState, ExtensionFileSource};
use crate::exec::{fixtures, runtime};
use crate::reporting::{CompilerError, InterpreterError, Problem};
use deno_core::{v8, Extension, OpState};
use elm_project_utils::{setup_generator_project, ElmPostProcessor, ElmResult}; use elm_project_utils::{setup_generator_project, ElmPostProcessor, ElmResult};
use os_pipe::dup_stderr; use os_pipe::dup_stderr;
use sqlx::sqlite::SqlitePool; use sqlx::sqlite::SqlitePool;
use std::borrow::Cow; use std::{
use std::cell::RefCell; borrow::Cow,
use std::convert::TryFrom; cell::RefCell,
use std::fs; convert::TryFrom,
use std::io; fs,
use std::io::Write; io,
use std::path::PathBuf; io::Write,
use std::process::{Command, Stdio}; path::PathBuf,
use std::sync::Arc; process::{Command, Stdio},
use tokio; sync::Arc,
};
use tracing::info_span; use tracing::info_span;
use crate::exec::fixtures::astrid_pages::ScriptError;
use crate::exec::{fixtures, runtime};
use crate::reporting::{CompilerError, InterpreterError, Problem};
#[derive(Debug, Copy, Clone)] #[derive(Debug, Copy, Clone)]
pub enum OutputType { pub enum OutputType {
Html, Html,
@ -156,21 +158,6 @@ pub(crate) fn run(
} }
_ => (), _ => (),
} }
// I think that when I set this script to be the main module, I am skipping the
// deno/runtime/js/99_main.js script that sets up a bunch of global variables. If I
// manually add the timer related code below then setTimeout works again.
// NB. there are 706 lines of setup code that add a bunch of apis to the global window
// scope. Figure out if I need to include all of them. For example, starmelon does not need
// to perform http calls right now, but I eventually want to.
buffer.push_str(
r#"
const { setTimeout } = globalThis.__bootstrap.timers;
Deno.core.setMacrotaskCallback(globalThis.__bootstrap.timers.handleTimerMacrotask);
globalThis.setTimeout = setTimeout;
"#,
);
buffer.push_str(&format!( buffer.push_str(&format!(
"var worker = Elm.{}.init({{flags: {{ stagename: \"Atago\"}} }});\n", "var worker = Elm.{}.init({{flags: {{ stagename: \"Atago\"}} }});\n",
@ -183,7 +170,7 @@ pub(crate) fn run(
globalThis.runOnInput = function(route) { worker.ports.onRequest.send(route) }; globalThis.runOnInput = function(route) { worker.ports.onRequest.send(route) };
worker.ports.onStringOutput.subscribe(function(result) { worker.ports.onStringOutput.subscribe(function(result) {
Deno.core.opSync('op_starmelon_string_output', result); Extension.starmelon_string_output(result);
}); });
// Elm will send a DataView // Elm will send a DataView
if (worker.ports.onBytesOutput) { if (worker.ports.onBytesOutput) {
@ -192,7 +179,7 @@ pub(crate) fn run(
const ui8 = new Uint8Array(result.a.buffer); const ui8 = new Uint8Array(result.a.buffer);
output.a = ui8; output.a = ui8;
} }
Deno.core.opSync('op_starmelon_bytes_output', result) Extension.starmelon_bytes_output(result)
}); });
}"#, }"#,
); );
@ -268,6 +255,36 @@ pub(crate) fn run(
op_state_fn: Some(Box::new(move |state| { op_state_fn: Some(Box::new(move |state| {
state.put(Arc::clone(&mailbox_clone)); state.put(Arc::clone(&mailbox_clone));
})), })),
esm_files: {
const JS: &'static [ExtensionFileSource] = &[ExtensionFileSource::new(
"ext:io/bootstrap.js",
{
const STR: ::deno_core::v8::OneByteConst = ::deno_core::FastStaticString::create_external_onebyte_const(
r#"
import { op_starmelon_bytes_output, op_starmelon_string_output, op_starmelon_problem } from "ext:core/ops";
function starmelon_bytes_output(msg) {
return op_starmelon_bytes_output(msg);
}
function starmelon_string_output(msg) {
return op_starmelon_string_output(msg);
}
function starmelon_problem(msg) {
return op_starmelon_problem(msg);
}
globalThis.Extension = Object.assign(
globalThis.Extension || {},
{ starmelon_bytes_output, starmelon_string_output, starmelon_problem }
);
"#
.as_bytes(),
);
let s: &'static ::deno_core::v8::OneByteConst = &STR;
::deno_core::FastStaticString::new(s)
},
)];
Cow::Borrowed(JS)
},
esm_entry_point: Some("ext:io/bootstrap.js"),
..Default::default() ..Default::default()
}]; }];
@ -342,10 +359,10 @@ pub(crate) fn run(
type OutputMailbox = Arc<RefCell<Option<Result<Vec<u8>, ScriptError>>>>; type OutputMailbox = Arc<RefCell<Option<Result<Vec<u8>, ScriptError>>>>;
#[deno_core::op] #[deno_core::op2]
fn op_starmelon_problem( fn op_starmelon_problem(
state: &mut deno_core::OpState, state: &mut OpState,
msg: ScriptError, #[serde] msg: ScriptError,
) -> Result<(), deno_core::error::AnyError> { ) -> Result<(), deno_core::error::AnyError> {
eprintln!("got problem from v8 runtime {:?}", &msg); eprintln!("got problem from v8 runtime {:?}", &msg);
let mailbox_clone = state.borrow::<OutputMailbox>(); let mailbox_clone = state.borrow::<OutputMailbox>();
@ -355,10 +372,10 @@ fn op_starmelon_problem(
Ok(()) Ok(())
} }
#[deno_core::op] #[deno_core::op2]
fn op_starmelon_bytes_output( fn op_starmelon_bytes_output(
state: &mut OpState, state: &mut OpState,
msg: ElmResult<deno_core::JsBuffer, ScriptError>, #[serde] msg: ElmResult<deno_core::JsBuffer, ScriptError>,
) -> Result<(), deno_core::error::AnyError> { ) -> Result<(), deno_core::error::AnyError> {
let mailbox_clone = state.borrow::<OutputMailbox>(); let mailbox_clone = state.borrow::<OutputMailbox>();
if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() { if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() {
@ -375,10 +392,10 @@ fn op_starmelon_bytes_output(
Ok(()) Ok(())
} }
#[deno_core::op] #[deno_core::op2]
fn op_starmelon_string_output( fn op_starmelon_string_output(
state: &mut OpState, state: &mut OpState,
msg: ElmResult<String, ScriptError>, #[serde] msg: ElmResult<String, ScriptError>,
) -> Result<(), deno_core::error::AnyError> { ) -> Result<(), deno_core::error::AnyError> {
let mailbox_clone = state.borrow::<OutputMailbox>(); let mailbox_clone = state.borrow::<OutputMailbox>();
if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() { if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() {

47
src/exec/css_in_elm.rs
View file

@ -1,6 +1,6 @@
use crate::exec::{fixtures, runtime}; use crate::exec::{fixtures, runtime};
use crate::reporting::{CompilerError, InterpreterError, Problem}; use crate::reporting::{CompilerError, InterpreterError, Problem};
use deno_core::{Extension, OpState}; use deno_core::{Extension, ExtensionFileSource, OpState};
use elm_project_utils::{setup_generator_project, ElmPostProcessor}; use elm_project_utils::{setup_generator_project, ElmPostProcessor};
use os_pipe::dup_stderr; use os_pipe::dup_stderr;
use serde::Deserialize; use serde::Deserialize;
@ -11,7 +11,6 @@ use std::io::Write;
use std::path::PathBuf; use std::path::PathBuf;
use std::process::{Command, Stdio}; use std::process::{Command, Stdio};
use std::sync::Arc; use std::sync::Arc;
use tokio;
use tracing::info_span; use tracing::info_span;
pub(crate) fn run( pub(crate) fn run(
@ -116,21 +115,6 @@ pub(crate) fn run(
} }
_ => (), _ => (),
} }
// I think that when I set this script to be the main module, I am skipping the
// deno/runtime/js/99_main.js script that sets up a bunch of global variables. If I
// manually add the timer related code below then setTimeout works again.
// NB. there are 706 lines of setup code that add a bunch of apis to the global window
// scope. Figure out if I need to include all of them. For example, starmelon does not need
// to perform http calls right now, but I eventually want to.
buffer.push_str(
r#"
const { setTimeout } = globalThis.__bootstrap.timers;
Deno.core.setMacrotaskCallback(globalThis.__bootstrap.timers.handleTimerMacrotask);
globalThis.setTimeout = setTimeout;
"#,
);
buffer.push_str(&format!("var worker = Elm.{}.init();\n", &gen_module_name)); buffer.push_str(&format!("var worker = Elm.{}.init();\n", &gen_module_name));
// add a shortcut for invoking the function so I don't have to traverse so many object // add a shortcut for invoking the function so I don't have to traverse so many object
@ -139,7 +123,7 @@ pub(crate) fn run(
r#" r#"
if (worker.ports.onFilesOutput) { if (worker.ports.onFilesOutput) {
worker.ports.onFilesOutput.subscribe(function(result){ worker.ports.onFilesOutput.subscribe(function(result){
Deno.core.opSync('op_starmelon_elm_css_files_output', result) Extension.starmelon_elm_css_files_output(result)
}); });
}"#, }"#,
); );
@ -180,6 +164,29 @@ pub(crate) fn run(
op_state_fn: Some(Box::new(move |state| { op_state_fn: Some(Box::new(move |state| {
state.put(Arc::clone(&mailbox_clone)); state.put(Arc::clone(&mailbox_clone));
})), })),
esm_files: {
const JS: &'static [ExtensionFileSource] =
&[ExtensionFileSource::new("ext:io/bootstrap.js", {
const STR: ::deno_core::v8::OneByteConst =
::deno_core::FastStaticString::create_external_onebyte_const(
r#"
import { op_starmelon_elm_css_files_output } from "ext:core/ops";
function starmelon_elm_css_files_output(msg) {
return op_starmelon_elm_css_files_output(msg);
}
globalThis.Extension = Object.assign(
globalThis.Extension || {},
{ starmelon_elm_css_files_output }
);
"#
.as_bytes(),
);
let s: &'static ::deno_core::v8::OneByteConst = &STR;
::deno_core::FastStaticString::new(s)
})];
Cow::Borrowed(JS)
},
esm_entry_point: Some("ext:io/bootstrap.js"),
..Default::default() ..Default::default()
}]; }];
@ -241,10 +248,10 @@ struct FileDefinition {
success: bool, success: bool,
} }
#[deno_core::op] #[deno_core::op2]
fn op_starmelon_elm_css_files_output( fn op_starmelon_elm_css_files_output(
state: &mut OpState, state: &mut OpState,
msg: Vec<FileDefinition>, #[serde] msg: Vec<FileDefinition>,
) -> Result<(), deno_core::error::AnyError> { ) -> Result<(), deno_core::error::AnyError> {
let mailbox_clone = state.borrow::<OutputMailbox>(); let mailbox_clone = state.borrow::<OutputMailbox>();
if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() { if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() {

28
src/exec/fixtures/astrid_pages.rs
View file

@ -32,33 +32,33 @@ pub(crate) fn generate(
import Platform import Platform
-- START CUSTOMIZED PART -- START CUSTOMIZED PART
import #(flags_module) exposing (Flags) import $(flags_module) exposing (Flags)
#(for (target_module, _) in target_modules.iter() => $(for (target_module, _) in target_modules.iter() =>
import #(target_module) import $(target_module)
#<push> $<push>
) )
dispatch route flags = dispatch route flags =
case route of case route of
#(for (target_module, output_type) in target_modules.iter() => $(for (target_module, output_type) in target_modules.iter() =>
#(" ")#(quoted(target_module)) -> $(" ")$(quoted(target_module)) ->
#( match output_type { $( match output_type {
OutputType::String => { OutputType::String => {
#(" ")evalRoute (onStringOutput << encodeString) flags #(target_module).route $(" ")evalRoute (onStringOutput << encodeString) flags $(target_module).route
} }
OutputType::Value => { OutputType::Value => {
#(" ")evalRoute (onStringOutput << encodeJson) flags #(target_module).route $(" ")evalRoute (onStringOutput << encodeJson) flags $(target_module).route
} }
OutputType::Bytes => { OutputType::Bytes => {
#(" ")evalRoute (onBytesOutput << encodeBytes) flags #(target_module).route $(" ")evalRoute (onBytesOutput << encodeBytes) flags $(target_module).route
} }
OutputType::Html => { OutputType::Html => {
#(" ")evalRoute (onStringOutput << encodeHtml) flags #(target_module).route $(" ")evalRoute (onStringOutput << encodeHtml) flags $(target_module).route
} }
}) })
#<push> $<push>
) )
#<line> $<line>
_ -> _ ->
onStringOutput (encodeFailure NotFound) onStringOutput (encodeFailure NotFound)
@ -198,7 +198,7 @@ pub(crate) fn generate(
in in
case case
D.decodeString D.decodeString
(decodeElmHtml ( #("\\")taggers eventHandler -> D.succeed ())) (decodeElmHtml ( $("\\")taggers eventHandler -> D.succeed ()))
(asJsonString node) (asJsonString node)
of of
Ok elmHtml -> Ok elmHtml ->

6
src/exec/fixtures/css_in_elm.rs
View file

@ -8,9 +8,9 @@ pub(crate) fn generate(source_checksum: u64, entrypoint: &elmi::Global) -> (Stri
import Stylesheets import Stylesheets
-- START CUSTOMIZED PART -- START CUSTOMIZED PART
import #(&entrypoint.0.module) import $(&entrypoint.0.module)
entrypoint = #(&entrypoint.0.module).#(&entrypoint.1) entrypoint = $(&entrypoint.0.module).$(&entrypoint.1)
-- END CUSTOMIZED PART -- END CUSTOMIZED PART
-- MAIN -- MAIN
@ -39,7 +39,7 @@ pub(crate) fn generate(source_checksum: u64, entrypoint: &elmi::Global) -> (Stri
structure = structure =
Css.File.toFileStructure <| Css.File.toFileStructure <|
List.map List.map
(#("\\")(fileName, stylesheets) -> ($("\\")(fileName, stylesheets) ->
(fileName, Css.File.compile stylesheets) (fileName, Css.File.compile stylesheets)
) )
entrypoint entrypoint

4
src/exec/fixtures/scripting.rs
View file

@ -180,14 +180,14 @@ decodeString value =
encodeFailure : String -> E.Value encodeFailure : String -> E.Value
encodeFailure msg = encodeFailure msg =
E.object E.object
[ ("ctor", E.string "Err") [ ("$", E.string "Err")
, ("a", E.string msg) , ("a", E.string msg)
] ]
encodeSuccess : E.Value -> E.Value encodeSuccess : E.Value -> E.Value
encodeSuccess value = encodeSuccess value =
E.object E.object
[ ("ctor", E.string "Ok") [ ("$", E.string "Ok")
, ("a", value) , ("a", value)
] ]

18
src/exec/fixtures/sql-client-integration.js
View file

@ -1,18 +1,18 @@
function (query) { var $author$project$Astrid$Query$execute = function (query) {
return $author$project$Astrid$Query$dummyExecute; return $author$project$Astrid$Query$dummyExecute;
}; };
// CORE QUERIES // CORE QUERIES
function __Debug_print(object) { function __Debug_print(object) {
Deno.core.print(JSON.stringify(object)); Deno[Deno.internal].core.print(JSON.stringify(object));
Deno.core.print("\n"); Deno[Deno.internal].core.print("\n");
} }
function __Debug_print_slots(values) { function __Debug_print_slots(values) {
var len = values.length; var len = values.length;
for (var i = 0; i < len; i++) { for (var i = 0; i < len; i++) {
Deno.core.print([" ", i, ": ", JSON.stringify(values[i]), "\n"].join("")); Deno[Deno.internal].core.print([" ", i, ": ", JSON.stringify(values[i]), "\n"].join(""));
} }
} }
@ -129,6 +129,7 @@ function _Query_runDecoder(decoder, sql, xs)
return $elm$core$Result$Ok($elm$core$Maybe$Nothing); return $elm$core$Result$Ok($elm$core$Maybe$Nothing);
} }
var result = _Json_runOnString.f(decoder.a, xs[0]); var result = _Json_runOnString.f(decoder.a, xs[0]);
console.log("result of decoding string", result)
if (!$elm$core$Result$isOk(result)) if (!$elm$core$Result$isOk(result))
{ {
@ -207,6 +208,8 @@ function _Query_runDecoder(decoder, sql, xs)
var _Query_execute = function(query) var _Query_execute = function(query)
{ {
try {
console.log("inside _Query_execute")
// queries: Array (Int, Query a) // queries: Array (Int, Query a)
// values: Array (Maybe a) // values: Array (Maybe a)
// callbacks: Array (Int, Fn: * -> a) // callbacks: Array (Int, Fn: * -> a)
@ -262,8 +265,8 @@ var _Query_execute = function(query)
} }
if (statements.length > 0) { if (statements.length > 0) {
var queryResult = Deno.core.opSync( console.log("Extension", Extension)
'op_starmelon_batch_queries', var queryResult = Extension.starmelon_batch_queries(
statements, statements,
); );
// I am assuming here that the Rust code is serializing the same // I am assuming here that the Rust code is serializing the same
@ -335,6 +338,9 @@ var _Query_execute = function(query)
return $elm$core$Result$Ok(values.pop().a) return $elm$core$Result$Ok(values.pop().a)
} }
} }
} catch(err) {
console.error("Failure:", err)
}
}; };
var $author$project$Astrid$Query$execute = _Query_execute; var $author$project$Astrid$Query$execute = _Query_execute;

32
src/exec/fixtures/sqlite.rs
View file

@ -1,5 +1,5 @@
use deno_core::futures::StreamExt; use deno_core::futures::StreamExt;
use deno_core::{Extension, Op, OpState}; use deno_core::{Extension, ExtensionFileSource, Op, OpState};
use elm_project_utils::ElmResult; use elm_project_utils::ElmResult;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use serde_json::{value::Map, value::Number, Value}; use serde_json::{value::Map, value::Number, Value};
@ -7,7 +7,6 @@ use sqlx::{sqlite::SqlitePool, Column, Row, TypeInfo, ValueRef};
use std::borrow::Cow; use std::borrow::Cow;
use std::thread::JoinHandle; use std::thread::JoinHandle;
use std::time::Instant; use std::time::Instant;
use tokio;
use tracing::{info_span, Instrument}; use tracing::{info_span, Instrument};
type SQLWorkerMailbox = std::sync::mpsc::Sender<( type SQLWorkerMailbox = std::sync::mpsc::Sender<(
@ -113,20 +112,45 @@ pub(crate) fn init(db_pool: SqlitePool) -> Result<(Extension, JoinHandle<()>), (
}); });
let worker_mailbox_clone = worker_mailbox.clone(); let worker_mailbox_clone = worker_mailbox.clone();
let extension = Extension { let extension = Extension {
name: "sqlite",
ops: Cow::Owned(vec![op_starmelon_batch_queries::DECL]), ops: Cow::Owned(vec![op_starmelon_batch_queries::DECL]),
op_state_fn: Some(Box::new(move |state| { op_state_fn: Some(Box::new(move |state| {
state.put(worker_mailbox_clone.clone()); state.put(worker_mailbox_clone.clone());
})), })),
esm_files: {
const JS: &'static [ExtensionFileSource] =
&[ExtensionFileSource::new("ext:sqlite/bootstrap.js", {
const STR: ::deno_core::v8::OneByteConst =
::deno_core::FastStaticString::create_external_onebyte_const(
r#"
import { op_starmelon_batch_queries } from "ext:core/ops";
function starmelon_batch_queries(queries) {
return op_starmelon_batch_queries(queries);
}
globalThis.Extension = Object.assign(globalThis.Extension || {}, { starmelon_batch_queries });
"#
.as_bytes(),
);
let s: &'static ::deno_core::v8::OneByteConst = &STR;
::deno_core::FastStaticString::new(s)
})];
Cow::Borrowed(JS)
},
esm_entry_point: Some("ext:sqlite/bootstrap.js"),
enabled: true,
..Default::default() ..Default::default()
}; };
Ok((extension, sql_worker_thread)) Ok((extension, sql_worker_thread))
} }
#[deno_core::op] deno_core::extension!(hello_runtime, ops = [op_starmelon_batch_queries]);
#[deno_core::op2]
#[serde]
fn op_starmelon_batch_queries( fn op_starmelon_batch_queries(
state: &mut OpState, state: &mut OpState,
queries: Vec<(bool, String, Vec<String>)>, #[serde] queries: Vec<(bool, String, Vec<String>)>,
) -> Result<ElmResult<Vec<Vec<String>>, AstridQueryError>, deno_core::error::AnyError> { ) -> Result<ElmResult<Vec<Vec<String>>, AstridQueryError>, deno_core::error::AnyError> {
let worker_mailbox_clone = state.borrow::<SQLWorkerMailbox>(); let worker_mailbox_clone = state.borrow::<SQLWorkerMailbox>();

57
src/exec/mod.rs
View file

@ -8,7 +8,7 @@ use tracing::info_span;
mod astrid_pages; mod astrid_pages;
mod css_in_elm; mod css_in_elm;
mod fixtures; pub(crate) mod fixtures;
mod scripting; mod scripting;
pub(crate) fn exec( pub(crate) fn exec(
@ -169,55 +169,49 @@ fn is_css_in_elm_stylesheet(tipe: &elmi::Type) -> bool {
false false
} }
mod runtime { pub(crate) mod runtime {
use crate::reporting::InterpreterError; use crate::reporting::InterpreterError;
use deno_core::error::{type_error, AnyError}; use deno_core::error::AnyError;
use deno_core::futures::FutureExt; use deno_core::{Extension, FsModuleLoader, ModuleSpecifier};
use deno_core::{resolve_url, Extension, FsModuleLoader, ModuleLoader, ModuleSpecifier};
use deno_runtime::deno_broadcast_channel::InMemoryBroadcastChannel;
use deno_runtime::permissions::PermissionsContainer; use deno_runtime::permissions::PermissionsContainer;
use deno_runtime::worker::MainWorker; use deno_runtime::worker::MainWorker;
use deno_runtime::worker::WorkerOptions; use deno_runtime::worker::WorkerOptions;
use deno_runtime::BootstrapOptions; //use deno_runtime::BootstrapOptions;
use deno_web::BlobStore;
use std::pin::Pin;
use std::rc::Rc; use std::rc::Rc;
use std::sync::Arc;
use tracing::{info_span, Instrument}; use tracing::{info_span, Instrument};
fn get_error_class_name(e: &AnyError) -> &'static str {
deno_runtime::errors::get_error_class_name(e).unwrap_or("Error")
}
pub fn setup_worker( pub fn setup_worker(
extensions: Vec<Extension>, extensions: Vec<Extension>,
path_str: &str, path_str: &str,
) -> Result<(MainWorker, ModuleSpecifier), AnyError> { ) -> Result<(MainWorker, ModuleSpecifier), AnyError> {
let module_loader = Rc::new(FsModuleLoader); //let module_loader = Rc::new(FsModuleLoader);
let options = WorkerOptions { let options = WorkerOptions {
bootstrap: BootstrapOptions { //module_loader: Rc::new(FsModuleLoader),
args: vec![], //bootstrap: BootstrapOptions {
cpu_count: 1, // args: vec![],
enable_testing_features: false, // cpu_count: 1,
location: None, // enable_testing_features: false,
no_color: false, // location: None,
is_tty: false, // no_color: false,
runtime_version: "0.127.0".to_string(), // is_tty: false,
ts_version: "2.0.0".to_string(), // runtime_version: "0.127.0".to_string(),
unstable: false, // ts_version: "2.0.0".to_string(),
user_agent: "starmelon".to_string(), // unstable: false,
..Default::default() // user_agent: "starmelon".to_string(),
}, // ..Default::default()
//},
extensions, extensions,
//should_break_on_first_statement: false,
//should_wait_for_inspector_session: false,
..Default::default() ..Default::default()
}; };
let main_module = ModuleSpecifier::from_file_path(path_str) let main_module = ModuleSpecifier::from_file_path(path_str)
.map_err(|_| AnyError::msg("path is not absolute"))?; .map_err(|()| AnyError::msg("path is not absolute"))?;
let permissions = PermissionsContainer::allow_all(); let permissions = PermissionsContainer::allow_all();
let worker = MainWorker::from_options(main_module.clone(), permissions, options); let worker = MainWorker::bootstrap_from_options(main_module.clone(), permissions, options);
//worker.bootstrap(&options); //worker.bootstrap(&options);
Ok((worker, main_module)) Ok((worker, main_module))
@ -231,12 +225,13 @@ mod runtime {
where where
F: FnOnce(deno_core::v8::HandleScope) -> Result<(), InterpreterError>, F: FnOnce(deno_core::v8::HandleScope) -> Result<(), InterpreterError>,
{ {
let wait_for_inspector = false;
// step 10 load the module into our v8 isolate // step 10 load the module into our v8 isolate
worker worker
.execute_main_module(&main_module) .execute_main_module(&main_module)
.instrument(info_span!("execute main module")) .instrument(info_span!("execute main module"))
.await?; .await?;
let wait_for_inspector = false;
worker worker
.run_event_loop(wait_for_inspector) .run_event_loop(wait_for_inspector)
.instrument(info_span!("run v8 event loop")) .instrument(info_span!("run v8 event loop"))

105
src/exec/scripting.rs
View file

@ -1,6 +1,6 @@
use crate::exec::{fixtures, runtime}; use crate::exec::{fixtures, runtime};
use crate::reporting::{CompilerError, InterpreterError, Problem, TypeError}; use crate::reporting::{CompilerError, InterpreterError, Problem, TypeError};
use deno_core::{Extension, OpState}; use deno_core::{Extension, ExtensionFileSource, OpState};
use elm_project_utils::{setup_generator_project, ElmResult}; use elm_project_utils::{setup_generator_project, ElmResult};
use os_pipe::dup_stderr; use os_pipe::dup_stderr;
use sqlx::sqlite::SqlitePool; use sqlx::sqlite::SqlitePool;
@ -12,7 +12,6 @@ use std::io::{self, Read, Write};
use std::path::PathBuf; use std::path::PathBuf;
use std::process::{Command, Stdio}; use std::process::{Command, Stdio};
use std::sync::Arc; use std::sync::Arc;
use tokio;
use tracing::info_span; use tracing::info_span;
pub(crate) fn run( pub(crate) fn run(
@ -66,6 +65,7 @@ pub(crate) fn run(
command command
.arg("make") .arg("make")
.arg("--report=json")
.arg("--output") .arg("--output")
.arg(&intermediate_file) .arg(&intermediate_file)
.current_dir(&generator_dir) .current_dir(&generator_dir)
@ -89,10 +89,13 @@ pub(crate) fn run(
match command.output() { match command.output() {
Ok(output) => { Ok(output) => {
if !output.status.success() { if !output.status.success() {
eprintln!("output of elm make: failed building scripting fixture file");
return Err(CompilerError::FailedBuildingFixture.into()); return Err(CompilerError::FailedBuildingFixture.into());
} }
//command.stdout(Stdio::piped());
} }
Err(_) => { Err(err) => {
eprintln!("got this kind of failure {:?}", err);
return Err(CompilerError::FailedBuildingFixture.into()); return Err(CompilerError::FailedBuildingFixture.into());
} }
} }
@ -158,9 +161,8 @@ pub(crate) fn run(
// final_script.replace("var $author$project$Astrid$Query$run = ", "JSON.stringify(x)"); // final_script.replace("var $author$project$Astrid$Query$run = ", "JSON.stringify(x)");
final_script.push_str("\n\n"); final_script.push_str("\n\n");
//final_script.push_str(r#" //final_script.push_str(r#"
// Deno.core.print(JSON.stringify( // Deno[Deno.internal].core.print(JSON.stringify(
// Deno.core.opSync( // Extension.starmelon_batch_queries(
// 'op_starmelon_batch_queries',
// [ [true, "select json_object('id', id, 'foo', foo) from foobar", []] // [ [true, "select json_object('id', id, 'foo', foo) from foobar", []]
// , [false, "select json_object('id', id, 'foo', foo) from foobar", []] // , [false, "select json_object('id', id, 'foo', foo) from foobar", []]
// ] // ]
@ -170,17 +172,6 @@ pub(crate) fn run(
} }
final_script.push_str("\n\n"); final_script.push_str("\n\n");
// I think that when I set this script to be the main module, I am skipping the
// deno/runtime/js/99_main.js script that sets up a bunch of global variables. If I
// manually add the timer related code below then setTimeout works again.
// NB. there are 706 lines of setup code that add a bunch of apis to the global window
// scope. Figure out if I need to include all of them. For example, starmelon does not need
// to perform http calls right now, but I eventually want to.
final_script.push_str("const { setTimeout } = globalThis.__bootstrap.timers;\n");
final_script.push_str(
"Deno.core.setMacrotaskCallback(globalThis.__bootstrap.timers.handleTimerMacrotask);\n",
);
final_script.push_str("globalThis.setTimeout = setTimeout;\n");
final_script.push_str(&format!("var worker = Elm.{}.init();\n", &gen_module_name)); final_script.push_str(&format!("var worker = Elm.{}.init();\n", &gen_module_name));
// add a shortcut for invoking the function so I don't have to traverse so many object // add a shortcut for invoking the function so I don't have to traverse so many object
@ -216,11 +207,12 @@ pub(crate) fn run(
final_script.push_str( final_script.push_str(
r#" r#"
worker.ports.onOutput.subscribe(function(output){ worker.ports.onOutput.subscribe(function(output){
console.log("got output value")
if (output.ctor === "Ok") { if (output.ctor === "Ok") {
const json = JSON.stringify(output.a); const json = JSON.stringify(output.a);
Deno.core.opSync('op_starmelon_string_output', output); Extension.starmelon_string_output(output);
} else { } else {
Deno.core.opSync('op_starmelon_problem', output.a); Extension.starmelon_problem(output.a);
} }
}); });
"#, "#,
@ -231,11 +223,12 @@ pub(crate) fn run(
r#" r#"
// Elm will send a DataView // Elm will send a DataView
worker.ports.onOutput.subscribe(function(output){ worker.ports.onOutput.subscribe(function(output){
console.log("got output bytes")
if (output.ctor === "Ok") { if (output.ctor === "Ok") {
const ui8 = new Uint8Array(output.a.buffer); const ui8 = new Uint8Array(output.a.buffer);
Deno.core.opSync('op_starmelon_bytes_output', ui8); Extension.starmelon_bytes_output(ui8);
} else { } else {
Deno.core.opSync('op_starmelon_problem', output.a); Extension.starmelon_problem(output.a);
} }
}); });
"#, "#,
@ -244,12 +237,10 @@ pub(crate) fn run(
OutputType::String | OutputType::Html => { OutputType::String | OutputType::Html => {
final_script.push_str( final_script.push_str(
r#" r#"
worker.ports.onOutput.subscribe(function(output){ worker.ports.onOutput.subscribe(function outputCallback(output){
if (output.ctor === "Ok") { console.log("calling ports.onOutput callback for HTML", output)
Deno.core.opSync('op_starmelon_string_output', output.a); // type ElmResult = Ok string | Err string
} else { Extension.starmelon_string_output(output);
Deno.core.opSync('op_starmelon_problem', output.a);
}
}); });
"#, "#,
); );
@ -268,6 +259,16 @@ pub(crate) fn run(
.map_err(|io_err| CompilerError::WriteOutputFailed(io_err, final_file.clone()))?; .map_err(|io_err| CompilerError::WriteOutputFailed(io_err, final_file.clone()))?;
drop(timing_guard); drop(timing_guard);
// for debugging purposes we have the option to pause here
//if true {
// let mut line = String::new();
// eprintln!(
// "Paused after writing `{:?}`. Continue ?",
// final_file.to_string_lossy()
// );
// std::io::stdin().read_line(&mut line).unwrap();
//}
// Create a tokio runtime before registering ops so we can block on futures inside sync ops // Create a tokio runtime before registering ops so we can block on futures inside sync ops
let span = info_span!("create tokio runtime"); let span = info_span!("create tokio runtime");
let timing_guard = span.enter(); let timing_guard = span.enter();
@ -286,6 +287,7 @@ pub(crate) fn run(
let mailbox_clone = Arc::clone(&mailbox); let mailbox_clone = Arc::clone(&mailbox);
let mut extensions = vec![Extension { let mut extensions = vec![Extension {
name: "io",
ops: Cow::Owned(vec![ ops: Cow::Owned(vec![
op_starmelon_bytes_output::decl(), op_starmelon_bytes_output::decl(),
op_starmelon_string_output::decl(), op_starmelon_string_output::decl(),
@ -294,6 +296,36 @@ pub(crate) fn run(
op_state_fn: Some(Box::new(move |state| { op_state_fn: Some(Box::new(move |state| {
state.put(Arc::clone(&mailbox_clone)); state.put(Arc::clone(&mailbox_clone));
})), })),
esm_files: {
const JS: &'static [ExtensionFileSource] = &[ExtensionFileSource::new(
"ext:io/bootstrap.js",
{
const STR: ::deno_core::v8::OneByteConst = ::deno_core::FastStaticString::create_external_onebyte_const(
r#"
import { op_starmelon_bytes_output, op_starmelon_string_output, op_starmelon_problem } from "ext:core/ops";
function starmelon_bytes_output(msg) {
return op_starmelon_bytes_output(msg);
}
function starmelon_string_output(msg) {
return op_starmelon_string_output(msg);
}
function starmelon_problem(msg) {
return op_starmelon_problem(msg);
}
globalThis.Extension = Object.assign(
globalThis.Extension || {},
{ starmelon_bytes_output, starmelon_string_output, starmelon_problem }
);
"#
.as_bytes(),
);
let s: &'static ::deno_core::v8::OneByteConst = &STR;
::deno_core::FastStaticString::new(s)
},
)];
Cow::Borrowed(JS)
},
esm_entry_point: Some("ext:io/bootstrap.js"),
..Default::default() ..Default::default()
}]; }];
@ -309,6 +341,7 @@ pub(crate) fn run(
.unwrap(); .unwrap();
if let Ok((extension, thread_handle)) = fixtures::sqlite::init(db_pool) { if let Ok((extension, thread_handle)) = fixtures::sqlite::init(db_pool) {
eprintln!("started sqlite extension");
extensions.push(extension); extensions.push(extension);
Some(thread_handle) Some(thread_handle)
} else { } else {
@ -364,7 +397,7 @@ pub(crate) fn run(
} }
}; };
let foo = move |mut scope: deno_core::v8::HandleScope| -> Result<(), InterpreterError> { let entrypoint = move |mut scope: deno_core::v8::HandleScope| -> Result<(), InterpreterError> {
let scope = &mut scope; let scope = &mut scope;
let ctx = scope.get_current_context(); let ctx = scope.get_current_context();
let global = ctx.global(scope); let global = ctx.global(scope);
@ -429,7 +462,7 @@ pub(crate) fn run(
let span = info_span!("eval javascript"); let span = info_span!("eval javascript");
let timing_guard = span.enter(); let timing_guard = span.enter();
sys.block_on(async move { runtime::xyz(worker, main_module, foo).await })?; sys.block_on(async move { runtime::xyz(worker, main_module, entrypoint).await })?;
drop(timing_guard); drop(timing_guard);
// step 13 receive the callback // step 13 receive the callback
@ -467,10 +500,10 @@ pub(crate) fn run(
type OutputMailbox = Arc<RefCell<Option<Result<Vec<u8>, String>>>>; type OutputMailbox = Arc<RefCell<Option<Result<Vec<u8>, String>>>>;
#[deno_core::op] #[deno_core::op2(fast)]
fn op_starmelon_problem( fn op_starmelon_problem(
state: &mut deno_core::OpState, state: &mut OpState,
msg: String, #[string] msg: String,
) -> Result<(), deno_core::error::AnyError> { ) -> Result<(), deno_core::error::AnyError> {
eprintln!("got problem from v8 runtime {:?}", &msg); eprintln!("got problem from v8 runtime {:?}", &msg);
let mailbox_clone = state.borrow::<OutputMailbox>(); let mailbox_clone = state.borrow::<OutputMailbox>();
@ -480,10 +513,10 @@ fn op_starmelon_problem(
Ok(()) Ok(())
} }
#[deno_core::op] #[deno_core::op2]
fn op_starmelon_bytes_output( fn op_starmelon_bytes_output(
state: &mut OpState, state: &mut OpState,
msg: ElmResult<deno_core::JsBuffer, String>, #[serde] msg: ElmResult<deno_core::JsBuffer, String>,
) -> Result<(), deno_core::error::AnyError> { ) -> Result<(), deno_core::error::AnyError> {
let mailbox_clone = state.borrow::<OutputMailbox>(); let mailbox_clone = state.borrow::<OutputMailbox>();
if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() { if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() {
@ -500,10 +533,10 @@ fn op_starmelon_bytes_output(
Ok(()) Ok(())
} }
#[deno_core::op] #[deno_core::op2]
fn op_starmelon_string_output( fn op_starmelon_string_output(
state: &mut OpState, state: &mut OpState,
msg: ElmResult<String, String>, #[serde] msg: ElmResult<String, String>,
) -> Result<(), deno_core::error::AnyError> { ) -> Result<(), deno_core::error::AnyError> {
let mailbox_clone = state.borrow::<OutputMailbox>(); let mailbox_clone = state.borrow::<OutputMailbox>();
if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() { if let Ok(mut mailbox) = mailbox_clone.try_borrow_mut() {

133
src/main.rs
View file

@ -5,9 +5,10 @@ use std::hash::Hasher;
use std::io::{self, Write}; use std::io::{self, Write};
use std::path::PathBuf; use std::path::PathBuf;
use std::time::Instant; use std::time::Instant;
use structopt::StructOpt;
use tracing::info_span; use tracing::info_span;
use clap::Parser;
mod debug;
mod derive; mod derive;
mod elm; mod elm;
mod exec; mod exec;
@ -16,12 +17,46 @@ mod timings;
mod transpile; mod transpile;
fn main() { fn main() {
let args = Arguments::from_args(); let args = Arguments::parse();
let (span_subscriber, timing_report) = Timings::new(); let (span_subscriber, timing_report) = Timings::new();
tracing::subscriber::set_global_default(span_subscriber).unwrap(); tracing::subscriber::set_global_default(span_subscriber).unwrap();
match args { match args {
Arguments::Clean => {
let result = (|| -> Result<(), crate::reporting::Problem> {
let elm_project_dir = crate::elm::find_project_root("elm.json", "./")
.map_err(crate::reporting::CompilerError::MissingElmJson)?;
for our_temp_dir in [
elm_project_dir.join("elm-stuff").join("starmelon-ce7993"),
elm_project_dir.join("elm-stuff").join("starmelon-5d9ecc")
] {
for entry in walkdir::WalkDir::new(our_temp_dir).contents_first(true) {
if entry.is_err() {
eprintln!("{:?}", entry);
continue;
}
let entry = entry.unwrap();
// TODO implement an error message for failure to delete a file during clean
if entry.file_type().is_file() {
let _ = std::fs::remove_file(entry.path());
}
if entry.file_type().is_dir() {
let _ = std::fs::remove_dir(entry.path());
}
eprintln!("{}", entry.path().display());
}
}
Ok(())
})();
if let Err(problem) = result {
let span = info_span!("pretty print problem");
let timing_guard = span.enter();
eprintln!("{}", pretty(80, problem.to_doc()));
drop(timing_guard);
}
}
Arguments::Exec { Arguments::Exec {
file, file,
debug, debug,
@ -35,7 +70,7 @@ fn main() {
let start = Instant::now(); let start = Instant::now();
let span = info_span!("exec"); let span = info_span!("exec");
let timing_guard = span.enter(); let timing_guard = span.enter();
let result = exec::exec(file, debug, function, input, output, verbosity, sqlite); let result = exec::exec(file, debug, function, input, output, verbosity as u64, sqlite);
drop(timing_guard); drop(timing_guard);
if let Err(problem) = result { if let Err(problem) = result {
let span = info_span!("pretty print problem"); let span = info_span!("pretty print problem");
@ -69,8 +104,9 @@ fn main() {
verbosity, verbosity,
debug, debug,
} => { } => {
transpile::transpile(file, debug, function, verbosity).unwrap(); transpile::transpile(file, debug, function, verbosity.into()).unwrap();
} }
Arguments::Derive(DeriveMacros::Other) => (),
Arguments::Derive(DeriveMacros::LiveTable { Arguments::Derive(DeriveMacros::LiveTable {
file, file,
debug, debug,
@ -82,7 +118,7 @@ fn main() {
let span = info_span!("derive livetable editor"); let span = info_span!("derive livetable editor");
let timing_guard = span.enter(); let timing_guard = span.enter();
let result = let result =
derive::derive_livetable(file, debug, output, verbosity /*, sqlite */); derive::derive_livetable(file, debug, output, verbosity.into() /*, sqlite */);
drop(timing_guard); drop(timing_guard);
if let Err(problem) = result { if let Err(problem) = result {
let span = info_span!("pretty print problem"); let span = info_span!("pretty print problem");
@ -110,6 +146,40 @@ fn main() {
Instant::now() - start Instant::now() - start
); );
} }
Arguments::Debug { javascript, timings } => {
let start = Instant::now();
let span = info_span!("debug javascript");
let timing_guard = span.enter();
let result = crate::debug::run(javascript);
if let Err(problem) = result {
let span = info_span!("pretty print problem");
let timing_guard = span.enter();
eprintln!("{}", pretty(80, problem.to_doc()));
drop(timing_guard);
}
{
let stdout = io::stdout();
let mut handle = stdout.lock();
handle.flush().unwrap();
}
if timings {
let mut report = timing_report.dump().unwrap();
report.sort_by(|(_, a), (_, b)| b.partial_cmp(a).unwrap());
for (step, duration) in report.iter() {
eprintln!("[{:>10.3?}] {}", duration, step);
}
}
eprintln!(
"\t\x1b[1;92mFinished\x1b[0m [{}] in {:?}",
"unoptimized",
Instant::now() - start
);
}
} }
} }
@ -117,8 +187,9 @@ struct PortableHash(ahash::AHasher);
impl PortableHash { impl PortableHash {
fn new() -> Self { fn new() -> Self {
use std::hash::BuildHasher;
// We need constant keys to get the same checksup every time we run the program. // We need constant keys to get the same checksup every time we run the program.
Self(ahash::AHasher::new_with_keys(1, 2)) Self(ahash::RandomState::with_seeds(1, 2, 3, 4).build_hasher())
} }
} }
@ -143,53 +214,59 @@ impl ::core::hash::Hasher for PortableHash {
} }
} }
#[derive(Debug, StructOpt)] #[derive(Debug, clap::Parser)]
enum Arguments { enum Arguments {
Clean,
Exec { Exec {
#[structopt(parse(from_os_str))]
file: PathBuf, file: PathBuf,
function: String, function: String,
#[structopt(long)] #[arg(long)]
debug: bool, debug: bool,
#[structopt(long)] #[arg(long)]
input: Option<PathBuf>, input: Option<PathBuf>,
#[structopt(long)] #[arg(long)]
output: Option<PathBuf>, output: Option<PathBuf>,
#[structopt(short = "v", parse(from_occurrences))] #[arg(short = 'v', action(clap::ArgAction::Count))]
verbosity: u64, verbosity: u8,
#[structopt(long)] #[arg(long)]
sqlite: Option<PathBuf>, sqlite: Option<PathBuf>,
#[structopt(long)] #[arg(long)]
timings: bool, timings: bool,
}, },
Transpile { Transpile {
#[structopt(parse(from_os_str))]
file: PathBuf, file: PathBuf,
function: String, function: String,
#[structopt(short = "v", parse(from_occurrences))] #[arg(short = 'v', action(clap::ArgAction::Count))]
verbosity: u64, verbosity: u8,
#[structopt(long)] #[arg(long)]
debug: bool, debug: bool,
}, },
#[structopt(name = "derive")] #[command(name = "derive", subcommand)]
Derive(DeriveMacros), Derive(DeriveMacros),
Debug {
#[arg(long = "js")]
javascript: Option<PathBuf>,
#[arg(long)]
timings: bool,
}
} }
#[derive(Debug, StructOpt)] #[derive(Debug, clap::Parser)]
pub enum DeriveMacros { pub enum DeriveMacros {
#[structopt(name = "livetable")] #[command(name = "livetable")]
LiveTable { LiveTable {
#[structopt(parse(from_os_str))]
file: PathBuf, file: PathBuf,
#[structopt(short = "v", parse(from_occurrences))] #[arg(short = 'v', action(clap::ArgAction::Count))]
verbosity: u64, verbosity: u8,
#[structopt(long)] #[arg(long)]
debug: bool, debug: bool,
#[structopt(long)] #[arg(long)]
output: Option<PathBuf>, output: Option<PathBuf>,
#[structopt(long)] #[arg(long)]
timings: bool, timings: bool,
}, },
#[command(subcommand)]
Other,
} }
pub mod generated { pub mod generated {

9
src/timings.rs
View file

@ -1,8 +1,8 @@
//! This module provides a crude system for timing how long various steps of the program take to //! This module provides a crude system for timing how long various steps of the program take to
//! execute using the tracing framework. I seek absolution for suboptimal code. For example I want //! execute using the tracing framework. My goal is to give myself permission to write suboptimal
//! to throw a span around all clones and see exactly how long I spend copying data. If I see that //! code. For example I want to throw a span around all clones and see exactly how long I spend
//! I only spend a few milliseconds overall then that could be a price worth paying for development //! copying data. If I see that I only spend a few milliseconds overall then that could be a price
//! velocity. //! worth paying for development velocity.
//! //!
//! This is quite a different use case then flame graphs. //! This is quite a different use case then flame graphs.
use rustc_hash::FxHashMap; use rustc_hash::FxHashMap;
@ -16,7 +16,6 @@ use std::time::{Duration, Instant};
use tracing::span::{Attributes, Record}; use tracing::span::{Attributes, Record};
use tracing::{Event, Id, Metadata}; use tracing::{Event, Id, Metadata};
//
// Each step will be described with a less than 100 char string. Steps will overlap. // Each step will be described with a less than 100 char string. Steps will overlap.
// //
// I am not tracking the relationship between // I am not tracking the relationship between

184
src/transpile.rs
View file

@ -402,45 +402,45 @@ mod rust_codegen {
body: &elmi::Expr, body: &elmi::Expr,
) { ) {
quote_in! { *tokens => quote_in! { *tokens =>
fn #name#(if !type_variables.is_empty() => fn $name$(if !type_variables.is_empty() =>
<#(for tvar in type_variables.iter() join (, ) => <$(for tvar in type_variables.iter() join (, ) =>
#tvar $tvar
)> )>
)(#(for (parameter, tipe) in parameters.iter() join (, ) => )($(for (parameter, tipe) in parameters.iter() join (, ) =>
#parameter: #(ref out { codegen_type(out, tipe) }) $parameter: $(ref out { codegen_type(out, tipe) })
)) -> #(ref out { codegen_type(out, &return_type) }) { )) -> $(ref out { codegen_type(out, &return_type) }) {
#(ref out { codegen_expr(out, symbol_table, body) }) $(ref out { codegen_expr(out, symbol_table, body) })
} }
} }
} }
fn codegen_type(tokens: &mut rust::Tokens, tipe: &elmi::Type) { fn codegen_type(tokens: &mut rust::Tokens, tipe: &elmi::Type) {
quote_in! { *tokens => quote_in! { *tokens =>
#(match tipe { $(match tipe {
elmi::Type::TLambda(a, b) => { elmi::Type::TLambda(a, b) => {
( #(ref out => codegen_type(out, a) ) -> #(ref out => codegen_type(out, b) ) ) ( $(ref out => codegen_type(out, a) ) -> $(ref out => codegen_type(out, b) ) )
} }
elmi::Type::TVar(elmi::Name(variable)) => { elmi::Type::TVar(elmi::Name(variable)) => {
#variable $variable
}, },
elmi::Type::TType(module_name, name, args) if module_name == "elm/core/String" && name == "String" && args.is_empty() => { elmi::Type::TType(module_name, name, args) if module_name == "elm/core/String" && name == "String" && args.is_empty() => {
String String
} }
elmi::Type::TType(home, name, args) if args.is_empty() => { elmi::Type::TType(home, name, args) if args.is_empty() => {
#(ref out => codegen_name_from_global(out, home, name)) $(ref out => codegen_name_from_global(out, home, name))
} }
elmi::Type::TType(home, name, args) => { elmi::Type::TType(home, name, args) => {
#(ref out => codegen_name_from_global(out, home, name))<#(for arg in args join(, ) => $(ref out => codegen_name_from_global(out, home, name))<$(for arg in args join(, ) =>
#(ref out => codegen_type(out, arg)) $(ref out => codegen_type(out, arg))
)> )>
} }
// // Might be a primitive type // // Might be a primitive type
// #(if module_name == "elm/core/String" && name == "String" => String) // $(if module_name == "elm/core/String" && name == "String" => String)
// #(if module_name == "elm/core/Basics" && name == "Int" => i64) // $(if module_name == "elm/core/Basics" && name == "Int" => i64)
// #(if module_name == "elm/core/Basics" && name == "Float" => f64) // $(if module_name == "elm/core/Basics" && name == "Float" => f64)
// #(if module_name == "elm/core/Basics" && name == "Bool" => bool) // $(if module_name == "elm/core/Basics" && name == "Bool" => bool)
// #(if module_name == "elm/core/Maybe" && name == "Maybe" => Option<i32>) // $(if module_name == "elm/core/Maybe" && name == "Maybe" => Option<i32>)
// #(if module_name == "elm/bytes/Bytes" && name == "Bytes" => Vec<u8>) // $(if module_name == "elm/bytes/Bytes" && name == "Bytes" => Vec<u8>)
//} //}
//elmi::Type::TType(_, _, _) => Err(TypeError::CantEvalCustomType), //elmi::Type::TType(_, _, _) => Err(TypeError::CantEvalCustomType),
//elmi::Type::TRecord(_, _) => Err(TypeError::CantEvalRecord), //elmi::Type::TRecord(_, _) => Err(TypeError::CantEvalRecord),
@ -479,7 +479,7 @@ mod rust_codegen {
name: &elmi::Name, name: &elmi::Name,
) { ) {
quote_in! { *tokens => quote_in! { *tokens =>
#(ref out => codegen_home_to_builder(out, home) )__#name $(ref out => codegen_home_to_builder(out, home) )__$name
} }
} }
@ -490,7 +490,7 @@ mod rust_codegen {
} = global; } = global;
quote_in! { *tokens => quote_in! { *tokens =>
_#(author.replace("-", "_"))_#(project.replace("-", "_"))__#(home.0.replace(".", "_")) _$(author.replace("-", "_"))_$(project.replace("-", "_"))__$(home.0.replace(".", "_"))
} }
} }
@ -498,18 +498,18 @@ mod rust_codegen {
match expr { match expr {
elmi::Expr::Bool(true) => quote_in! { *tokens => true }, elmi::Expr::Bool(true) => quote_in! { *tokens => true },
elmi::Expr::Bool(false) => quote_in! { *tokens => false }, elmi::Expr::Bool(false) => quote_in! { *tokens => false },
elmi::Expr::Chr(c) => quote_in! { *tokens => #("'")#c#("'") }, elmi::Expr::Chr(c) => quote_in! { *tokens => $("'")$c$("'") },
elmi::Expr::Str(s) => quote_in! { *tokens => #(quoted(s)) }, elmi::Expr::Str(s) => quote_in! { *tokens => $(quoted(s)) },
elmi::Expr::Int(x) => quote_in! { *tokens => #(x.to_string()) }, elmi::Expr::Int(x) => quote_in! { *tokens => $(x.to_string()) },
elmi::Expr::Float(x) => quote_in! { *tokens => #(x.to_string()) }, elmi::Expr::Float(x) => quote_in! { *tokens => $(x.to_string()) },
elmi::Expr::VarLocal(name) => { elmi::Expr::VarLocal(name) => {
quote_in! { *tokens => quote_in! { *tokens =>
#name $name
} }
} }
elmi::Expr::VarGlobal(elmi::Global(home, name)) => { elmi::Expr::VarGlobal(elmi::Global(home, name)) => {
quote_in! { *tokens => quote_in! { *tokens =>
#(ref out => codegen_name_from_global(out, home, name)) $(ref out => codegen_name_from_global(out, home, name))
} }
} }
//elmi::Expr::VarEnum(Global, IndexZeroBased), //elmi::Expr::VarEnum(Global, IndexZeroBased),
@ -523,7 +523,7 @@ mod rust_codegen {
} else { } else {
quote_in! { *tokens => quote_in! { *tokens =>
&[ &[
#(for x in xs join (,#<push>) => #(ref out => codegen_expr(out, symbol_table, x) ) ) $(for x in xs join (,$<push>) => $(ref out => codegen_expr(out, symbol_table, x) ) )
] ]
} }
} }
@ -531,7 +531,7 @@ mod rust_codegen {
elmi::Expr::Function(_parameters, _body) => { elmi::Expr::Function(_parameters, _body) => {
quote_in! { *tokens => quote_in! { *tokens =>
"i don't know how to code gen a function expression" "i don't know how to code gen a function expression"
//#(for elmi::Name(ref parameter) in parameters.iter() join (, ) => //$(for elmi::Name(ref parameter) in parameters.iter() join (, ) =>
//) //)
} }
} }
@ -548,11 +548,11 @@ mod rust_codegen {
))); )));
} }
quote_in! { *tokens => quote_in! { *tokens =>
Box::new(| #(for arg in closure_args.iter() join (, ) => #(ref out => codegen_expr(out, symbol_table, arg))) | { Box::new(| $(for arg in closure_args.iter() join (, ) => $(ref out => codegen_expr(out, symbol_table, arg))) | {
#(ref out => { $(ref out => {
codegen_name_from_global(out, home, name) codegen_name_from_global(out, home, name)
})( })(
#(for arg in args.iter().chain(closure_args.iter()) join (,#<push>) => #(ref out => $(for arg in args.iter().chain(closure_args.iter()) join (,$<push>) => $(ref out =>
codegen_expr(out, symbol_table, arg) ) codegen_expr(out, symbol_table, arg) )
) )
) )
@ -560,10 +560,10 @@ mod rust_codegen {
} }
} else { } else {
quote_in! { *tokens => quote_in! { *tokens =>
#(ref out => { $(ref out => {
codegen_name_from_global(out, home, name) codegen_name_from_global(out, home, name)
})( })(
#(for arg in args join (,#<push>) => #(ref out => $(for arg in args join (,$<push>) => $(ref out =>
codegen_expr(out, symbol_table, arg) ) codegen_expr(out, symbol_table, arg) )
) )
) )
@ -581,8 +581,8 @@ mod rust_codegen {
} }
elmi::Expr::VarLocal(name) => { elmi::Expr::VarLocal(name) => {
quote_in! { *tokens => quote_in! { *tokens =>
#name( $name(
#(for arg in args join (,#<push>) => #(ref out => $(for arg in args join (,$<push>) => $(ref out =>
codegen_expr(out, symbol_table, arg) ) codegen_expr(out, symbol_table, arg) )
) )
) )
@ -593,7 +593,7 @@ mod rust_codegen {
// TODO write a function that can take an expression and return the arity using // TODO write a function that can take an expression and return the arity using
// the symbol table from the bottom up. // the symbol table from the bottom up.
quote_in! { *tokens => quote_in! { *tokens =>
#(format!("{:?}", fexpr)) $(format!("{:?}", fexpr))
} }
//panic!("calling an expression not yet supported"); //panic!("calling an expression not yet supported");
} }
@ -602,19 +602,19 @@ mod rust_codegen {
//elmi::Expr::TailCall(Name, Vec<(Name, Expr)>), //elmi::Expr::TailCall(Name, Vec<(Name, Expr)>),
elmi::Expr::If(branches, _final_branch) => { elmi::Expr::If(branches, _final_branch) => {
quote_in! { *tokens => quote_in! { *tokens =>
#(for (condition, expr) in branches join (#<push>#("} else")) => $(for (condition, expr) in branches join ($<push>$("} else")) =>
if #(ref out => codegen_expr(out, symbol_table, condition)) #("{") if $(ref out => codegen_expr(out, symbol_table, condition)) $("{")
#(ref out => codegen_expr(out, symbol_table, expr)) $(ref out => codegen_expr(out, symbol_table, expr))
) #("} else {") ) $("} else {")
#(ref out => codegen_expr(out, symbol_table, expr)) $(ref out => codegen_expr(out, symbol_table, expr))
#("}") $("}")
} }
} }
elmi::Expr::Let(def, expr) => { elmi::Expr::Let(def, expr) => {
quote_in! { *tokens => quote_in! { *tokens =>
#(ref out => codegen_def(out, symbol_table, def)) $(ref out => codegen_def(out, symbol_table, def))
#<push> $<push>
#(ref out => codegen_expr(out, symbol_table, expr)) $(ref out => codegen_expr(out, symbol_table, expr))
} }
} }
//elmi::Expr::Destruct(Destructor, Box<Expr>), //elmi::Expr::Destruct(Destructor, Box<Expr>),
@ -624,7 +624,7 @@ mod rust_codegen {
} }
elmi::Expr::Accessor(name) => { elmi::Expr::Accessor(name) => {
quote_in! { *tokens => quote_in! { *tokens =>
Box::new(|_v| { _v.#name }) Box::new(|_v| { _v.$name })
} }
} }
//elmi::Expr::Access(Box<Expr>, Name), //elmi::Expr::Access(Box<Expr>, Name),
@ -633,16 +633,16 @@ mod rust_codegen {
elmi::Expr::Unit => (), elmi::Expr::Unit => (),
elmi::Expr::Tuple(a, b, None) => { elmi::Expr::Tuple(a, b, None) => {
quote_in! { *tokens => quote_in! { *tokens =>
( #(ref out => codegen_expr(out, symbol_table, a) ), #(ref out => codegen_expr(out, symbol_table, b) ) ) ( $(ref out => codegen_expr(out, symbol_table, a) ), $(ref out => codegen_expr(out, symbol_table, b) ) )
} }
} }
elmi::Expr::Tuple(a, b, Some(c)) => { elmi::Expr::Tuple(a, b, Some(c)) => {
quote_in! { *tokens => quote_in! { *tokens =>
( #(ref out => codegen_expr(out, symbol_table, a) ), #(ref out => codegen_expr(out, symbol_table, b) ), #(ref out => codegen_expr(out, symbol_table, c) ) ) ( $(ref out => codegen_expr(out, symbol_table, a) ), $(ref out => codegen_expr(out, symbol_table, b) ), $(ref out => codegen_expr(out, symbol_table, c) ) )
} }
} }
//elmi::Expr::Shader(ShaderSource, HashSet<Name>, HashSet<Name>), //elmi::Expr::Shader(ShaderSource, HashSet<Name>, HashSet<Name>),
_ => quote_in! { *tokens => #(format!("{:?}", expr)) }, _ => quote_in! { *tokens => $(format!("{:?}", expr)) },
} }
} }
@ -650,14 +650,14 @@ mod rust_codegen {
match def { match def {
elmi::Def::Def(name, expr) => { elmi::Def::Def(name, expr) => {
quote_in! { *tokens => quote_in! { *tokens =>
let #name = #(ref out => codegen_expr(out, symbol_table, expr) ); let $name = $(ref out => codegen_expr(out, symbol_table, expr) );
} }
} }
elmi::Def::TailDef(name, arg_names, expr) => { elmi::Def::TailDef(name, arg_names, expr) => {
quote_in! { *tokens => quote_in! { *tokens =>
|#(for arg in arg_names join (, ) => mut #arg) | { |$(for arg in arg_names join (, ) => mut $arg) | {
#("'")#name : loop { $("'")$name : loop {
#(ref out => codegen_expr(out, symbol_table, expr)) $(ref out => codegen_expr(out, symbol_table, expr))
} }
} }
} }
@ -736,10 +736,10 @@ mod lua_codegen {
body: &elmi::Expr, body: &elmi::Expr,
) { ) {
quote_in! { *tokens => quote_in! { *tokens =>
function #name(#(for (parameter, _tipe) in parameters.iter() join (, ) => function $name($(for (parameter, _tipe) in parameters.iter() join (, ) =>
#parameter $parameter
)) ))
#(ref out { codegen_expr(out, symbol_table, body) }) $(ref out { codegen_expr(out, symbol_table, body) })
end end
} }
} }
@ -750,7 +750,7 @@ mod lua_codegen {
name: &elmi::Name, name: &elmi::Name,
) { ) {
quote_in! { *tokens => quote_in! { *tokens =>
#(ref out => codegen_home_to_builder(out, home) )__#name $(ref out => codegen_home_to_builder(out, home) )__$name
} }
} }
@ -761,7 +761,7 @@ mod lua_codegen {
} = global; } = global;
quote_in! { *tokens => quote_in! { *tokens =>
_#(author.replace("-", "_"))_#(project.replace("-", "_"))__#(home.0.replace(".", "_")) _$(author.replace("-", "_"))_$(project.replace("-", "_"))__$(home.0.replace(".", "_"))
} }
} }
@ -769,18 +769,18 @@ mod lua_codegen {
match expr { match expr {
elmi::Expr::Bool(true) => quote_in! { *tokens => true }, elmi::Expr::Bool(true) => quote_in! { *tokens => true },
elmi::Expr::Bool(false) => quote_in! { *tokens => false }, elmi::Expr::Bool(false) => quote_in! { *tokens => false },
elmi::Expr::Chr(c) => quote_in! { *tokens => #(quoted(c)) }, elmi::Expr::Chr(c) => quote_in! { *tokens => $(quoted(c)) },
elmi::Expr::Str(s) => quote_in! { *tokens => #(quoted(s)) }, elmi::Expr::Str(s) => quote_in! { *tokens => $(quoted(s)) },
elmi::Expr::Int(x) => quote_in! { *tokens => #(x.to_string()) }, elmi::Expr::Int(x) => quote_in! { *tokens => $(x.to_string()) },
elmi::Expr::Float(x) => quote_in! { *tokens => #(x.to_string()) }, elmi::Expr::Float(x) => quote_in! { *tokens => $(x.to_string()) },
elmi::Expr::VarLocal(name) => { elmi::Expr::VarLocal(name) => {
quote_in! { *tokens => quote_in! { *tokens =>
#name $name
} }
} }
elmi::Expr::VarGlobal(elmi::Global(home, name)) => { elmi::Expr::VarGlobal(elmi::Global(home, name)) => {
quote_in! { *tokens => quote_in! { *tokens =>
#(ref out => codegen_name_from_global(out, home, name)) $(ref out => codegen_name_from_global(out, home, name))
} }
} }
//elmi::Expr::VarEnum(Global, IndexZeroBased), //elmi::Expr::VarEnum(Global, IndexZeroBased),
@ -794,7 +794,7 @@ mod lua_codegen {
} else { } else {
quote_in! { *tokens => quote_in! { *tokens =>
&[ &[
#(for x in xs join (,#<push>) => #(ref out => codegen_expr(out, symbol_table, x) ) ) $(for x in xs join (,$<push>) => $(ref out => codegen_expr(out, symbol_table, x) ) )
] ]
} }
} }
@ -802,7 +802,7 @@ mod lua_codegen {
elmi::Expr::Function(_parameters, _body) => { elmi::Expr::Function(_parameters, _body) => {
quote_in! { *tokens => quote_in! { *tokens =>
"i don't know how to code gen a function expression" "i don't know how to code gen a function expression"
//#(for elmi::Name(ref parameter) in parameters.iter() join (, ) => //$(for elmi::Name(ref parameter) in parameters.iter() join (, ) =>
//) //)
} }
} }
@ -819,11 +819,11 @@ mod lua_codegen {
))); )));
} }
quote_in! { *tokens => quote_in! { *tokens =>
Box::new(| #(for arg in closure_args.iter() join (, ) => #(ref out => codegen_expr(out, symbol_table, arg))) | { Box::new(| $(for arg in closure_args.iter() join (, ) => $(ref out => codegen_expr(out, symbol_table, arg))) | {
#(ref out => { $(ref out => {
codegen_name_from_global(out, home, name) codegen_name_from_global(out, home, name)
})( })(
#(for arg in args.iter().chain(closure_args.iter()) join (,#<push>) => #(ref out => $(for arg in args.iter().chain(closure_args.iter()) join (,$<push>) => $(ref out =>
codegen_expr(out, symbol_table, arg) ) codegen_expr(out, symbol_table, arg) )
) )
) )
@ -831,10 +831,10 @@ mod lua_codegen {
} }
} else { } else {
quote_in! { *tokens => quote_in! { *tokens =>
#(ref out => { $(ref out => {
codegen_name_from_global(out, home, name) codegen_name_from_global(out, home, name)
})( })(
#(for arg in args join (,#<push>) => #(ref out => $(for arg in args join (,$<push>) => $(ref out =>
codegen_expr(out, symbol_table, arg) ) codegen_expr(out, symbol_table, arg) )
) )
) )
@ -852,8 +852,8 @@ mod lua_codegen {
} }
elmi::Expr::VarLocal(name) => { elmi::Expr::VarLocal(name) => {
quote_in! { *tokens => quote_in! { *tokens =>
#name( $name(
#(for arg in args join (,#<push>) => #(ref out => $(for arg in args join (,$<push>) => $(ref out =>
codegen_expr(out, symbol_table, arg) ) codegen_expr(out, symbol_table, arg) )
) )
) )
@ -864,7 +864,7 @@ mod lua_codegen {
// TODO write a function that can take an expression and return the arity using // TODO write a function that can take an expression and return the arity using
// the symbol table from the bottom up. // the symbol table from the bottom up.
quote_in! { *tokens => quote_in! { *tokens =>
#(format!("{:?}", fexpr)) $(format!("{:?}", fexpr))
} }
//panic!("calling an expression not yet supported"); //panic!("calling an expression not yet supported");
} }
@ -873,19 +873,19 @@ mod lua_codegen {
//elmi::Expr::TailCall(Name, Vec<(Name, Expr)>), //elmi::Expr::TailCall(Name, Vec<(Name, Expr)>),
elmi::Expr::If(branches, _final_branch) => { elmi::Expr::If(branches, _final_branch) => {
quote_in! { *tokens => quote_in! { *tokens =>
#(for (condition, expr) in branches join (#<push>#("} else")) => $(for (condition, expr) in branches join ($<push>$("} else")) =>
if #(ref out => codegen_expr(out, symbol_table, condition)) #("{") if $(ref out => codegen_expr(out, symbol_table, condition)) $("{")
#(ref out => codegen_expr(out, symbol_table, expr)) $(ref out => codegen_expr(out, symbol_table, expr))
) #("} else {") ) $("} else {")
#(ref out => codegen_expr(out, symbol_table, expr)) $(ref out => codegen_expr(out, symbol_table, expr))
#("}") $("}")
} }
} }
elmi::Expr::Let(def, expr) => { elmi::Expr::Let(def, expr) => {
quote_in! { *tokens => quote_in! { *tokens =>
#(ref out => codegen_def(out, symbol_table, def)) $(ref out => codegen_def(out, symbol_table, def))
#<push> $<push>
#(ref out => codegen_expr(out, symbol_table, expr)) $(ref out => codegen_expr(out, symbol_table, expr))
} }
} }
//elmi::Expr::Destruct(Destructor, Box<Expr>), //elmi::Expr::Destruct(Destructor, Box<Expr>),
@ -896,7 +896,7 @@ mod lua_codegen {
} }
elmi::Expr::Accessor(name) => { elmi::Expr::Accessor(name) => {
quote_in! { *tokens => quote_in! { *tokens =>
Box::new(|_v| { _v.#name }) Box::new(|_v| { _v.$name })
} }
} }
//elmi::Expr::Access(Box<Expr>, Name), //elmi::Expr::Access(Box<Expr>, Name),
@ -905,16 +905,16 @@ mod lua_codegen {
elmi::Expr::Unit => (), elmi::Expr::Unit => (),
elmi::Expr::Tuple(a, b, None) => { elmi::Expr::Tuple(a, b, None) => {
quote_in! { *tokens => quote_in! { *tokens =>
( #(ref out => codegen_expr(out, symbol_table, a) ), #(ref out => codegen_expr(out, symbol_table, b) ) ) ( $(ref out => codegen_expr(out, symbol_table, a) ), $(ref out => codegen_expr(out, symbol_table, b) ) )
} }
} }
elmi::Expr::Tuple(a, b, Some(c)) => { elmi::Expr::Tuple(a, b, Some(c)) => {
quote_in! { *tokens => quote_in! { *tokens =>
( #(ref out => codegen_expr(out, symbol_table, a) ), #(ref out => codegen_expr(out, symbol_table, b) ), #(ref out => codegen_expr(out, symbol_table, c) ) ) ( $(ref out => codegen_expr(out, symbol_table, a) ), $(ref out => codegen_expr(out, symbol_table, b) ), $(ref out => codegen_expr(out, symbol_table, c) ) )
} }
} }
//elmi::Expr::Shader(ShaderSource, HashSet<Name>, HashSet<Name>), //elmi::Expr::Shader(ShaderSource, HashSet<Name>, HashSet<Name>),
_ => quote_in! { *tokens => #(format!("{:?}", expr)) }, _ => quote_in! { *tokens => $(format!("{:?}", expr)) },
} }
} }
@ -922,14 +922,14 @@ mod lua_codegen {
match def { match def {
elmi::Def::Def(name, expr) => { elmi::Def::Def(name, expr) => {
quote_in! { *tokens => quote_in! { *tokens =>
local #name = #(ref out => codegen_expr(out, symbol_table, expr) ) local $name = $(ref out => codegen_expr(out, symbol_table, expr) )
} }
} }
elmi::Def::TailDef(name, arg_names, expr) => { elmi::Def::TailDef(name, arg_names, expr) => {
quote_in! { *tokens => quote_in! { *tokens =>
|#(for arg in arg_names join (, ) => mut #arg) | { |$(for arg in arg_names join (, ) => mut $arg) | {
#("'")#name : loop { $("'")$name : loop {
#(ref out => codegen_expr(out, symbol_table, expr)) $(ref out => codegen_expr(out, symbol_table, expr))
} }
} }
} }

20
starmelon-binary.do
View file

@ -1,7 +1,15 @@
if test -f ../../target/release/starmelon; then BINARY_PATH="../../target/release/starmelon"
redo-ifchange always-rebuild
ls -al ../../../target/release/starmelon | redo-stamp redo-ifchange Cargo.toml
else; find src/ -type f | xargs redo-ifchange
cargo build --release find ../../../infra/rust-elmi/src -type f | xargs redo-ifchange
ls -al ../../../target/release/starmelon | redo-stamp find ../../../infra/redwood-lang/compiler/naive-wadler-prettier/src -type f | xargs redo-ifchange
find ../../../infra/rust-elm-project-utils/src -type f | xargs redo-ifchange
find ../../../infra/genco-extra/src -type f | xargs redo-ifchange
find ../../../infra/livetable/core/src -type f | xargs redo-ifchange
if [ ! -f "$BINARY_PATH" ]; then
cargo build --release --color=always
else
ls -al "$BINARY_PATH" | redo-stamp
fi fi